[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853757#comment-17853757
]
Oleg Kalnichevski commented on HTTPCLIENT-2330:
---
> And likely no way to avoid that.
[~doktorj] When using TLS one needs to stop counting seconds and to make sure
the connection does not get stuck for a minute or more.
However In your particular case you could run your application with TLS debug
on and see what protocol version gets negotiated and exactly what kind of
packets get exchanged between the two endpoints after the initial timeout.
Forcing older / newer version of TLS may help, or force-closing the connection
and foregoing graceful TLS layer shutdown may solve the problem for you.
Oleg
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853751#comment-17853751
]
Oleg Kalnichevski commented on HTTPCLIENT-2330:
---
[~doktorj] I suppose it is pretty simple. The TLS layer decides it needs to do
a handshake with the opposite endpoint that involves an extra read operation
(most likely the close notify) and there comes that extra socket timeout delay.
Oleg
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853744#comment-17853744
]
Oleg Kalnichevski commented on HTTPCLIENT-2330:
---
> We are scratching our heads over why it appears the flush is also taking a
> period of time equivalent to the socket timeout
[~doktorj] Are you using TLS over that connection?
Oleg
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2330.
---
Resolution: Invalid
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853737#comment-17853737
]
Oleg Kalnichevski edited comment on HTTPCLIENT-2330 at 6/10/24 4:57 PM:
> but this is very unexpected to have a socket timeout taking 2x the
> configured time (as if 2 separate timeouts are occurring).
[~doktorj] Moreover, Socket timeout is _not_ a deadline. It is perfectly normal
that the total message exchange time takes longer than the socket timeout
value, sometimes much longer, especially when TLS is being used. What you
likely want is HTTPCLIENT-1074, which all versions and all variants of
HttpClient presently do not support.
Oleg
was (Author: olegk):
> but this is very unexpected to have a socket timeout taking 2x the
> configured time (as if 2 separate timeouts are occurring).
[~doktorj] Moreover, Socket timeout is _not_ a deadline. It is perfectly normal
that the total message time takes longer than the socket timeout value,
sometimes much longer, especially when TLS is being used. What you likely want
is HTTPCLIENT-1074, which all versions and all variants of HttpClient presently
do not support.
Oleg
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853737#comment-17853737
]
Oleg Kalnichevski commented on HTTPCLIENT-2330:
---
> but this is very unexpected to have a socket timeout taking 2x the
> configured time (as if 2 separate timeouts are occurring).
[~doktorj] Moreover, Socket timeout is _not_ a deadline. It is perfectly normal
that the total message time takes longer than the socket timeout value,
sometimes much longer, especially when TLS is being used. What you likely want
is HTTPCLIENT-1074, which all versions and all variants of HttpClient presently
do not support.
Oleg
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2330) Socket timeouts taking 2x time requested
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17853716#comment-17853716
]
Oleg Kalnichevski commented on HTTPCLIENT-2330:
---
[~doktorj] I am not sure I understand the problem and what it is exactly you
consider a bug. Write operations on blocking sockets do not take the socket
timeout into consideration. We cannot change that.
Oleg
> Socket timeouts taking 2x time requested
>
>
> Key: HTTPCLIENT-2330
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2330
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Recreated on
> Unbuntu 23.10
> Java 17 and 21
>Reporter: John Woodward
>Priority: Major
>
> We are having an issue very similar to
> https://issues.apache.org/jira/browse/HTTPCLIENT-2324. We've had issues
> recreating this in a small unit test, however. We have traced through the
> execution of the code carefully, with trace-level logging enabled, and see
> the following:
> We have a socket timeout (responseTimeout) set to 5000 ms.
> We fire a request using RestTemplate to a point-of-sale endpoint using https
> and basic auth (Aloha; using a site like httpstat.us does not seem to
> recreate the issue, when using http or https).
> We see the outbound event at a timestamp such as
> {quote}2024-06-10T09:01:17.880-0500
> {quote}
> We see http.wire post the socket timeout at the expected time, 5 seconds
> after the initial request:
> {quote}2024-06-10T09:01:22.883-0500| DEBUG|
> http-nio-8080-exec-2| org.apache.hc.client5.http.wire|
> http-outgoing-2 << "[read] I/O error: Read timed out"
> 2024-06-10T09:01:22.883-0500| DEBUG| http-nio-8080-exec-2|
> http-outgoing-2 Close connection
> {quote}
> We then see the connection wait for an additional 5 seconds (adjusting the
> socket timeout, this time will be identical to the socket timeout), followed
> by:
> {quote}2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> endpoint closed
> 2024-06-10T09:01:27.889-0500| DEBUG|
> http-nio-8080-exec-2|nt5.http.impl.classic.InternalHttpClient| ep-03
> discarding endpoint
> {quote}
> Tracing with the debugger, the second delay appears to happen in
> org.apache.hc.core5.http.impl.io.BHttpConnectionBase, on line 255:
> {quote} this.outbuffer.flush(socketHolder.getOutputStream());
> {quote}
> Interestingly enough, if we are in the debugger we need to step over this
> line quickly to see it hang here for the socket timeout period; if we wait on
> this line, then the flush will happen quickly.
> I certainly understand the comments made in HTTPCLIENT-2324, that the socket
> timeout is just a timeout between events (so the connection cannot be idle
> longer that time span without encountering the socket timeout), but this
> appears to be clear that a socket timeout is being encountered, followed by a
> delay equivalent to the socket timeout timespan (another socket timeout being
> thrown?).
> A better way to handle an overall timeout would be to use something other
> than just a socket timeout — but this is very unexpected to have a socket
> timeout taking 2x the configured time (as if 2 separate timeouts are
> occurring).
> Btw, we have recreated this with the latest httpclient 4.x release, as well
> as 5.x, and we've used both java 17, and java 21.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-702) Improve connection limit guarantees
[
https://issues.apache.org/jira/browse/HTTPCORE-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCORE-702.
Resolution: Won't Fix
Closing due to inactivity.
Oleg
> Improve connection limit guarantees
> ---
>
> Key: HTTPCORE-702
> URL: https://issues.apache.org/jira/browse/HTTPCORE-702
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 4.4.14
> Environment: Ubuntu 18.04.6 LTS
> OpenJDK version 11.0.11
>Reporter: Nikola Bekcic
>Priority: Minor
> Labels: volunteers-wanted
>
> h3. Background
> To manage the life cycle and re-use HTTP connections we are using
> *PoolingHttpClientConnectionManager* with the maximum limit of connections
> defined on a per route basis as well as in total.
> Using PoolingHttpClientConnectionManager and built-in Java management
> utilities, I want to provide our users with the option to dynamically re-size
> the connection pool (up and/or down) at runtime without redeploying or
> restarting the application with a new configuration.
> h3. Issue description
> During testing, we found that decreasing the maximum limit of connection per
> route does not have (an immediate) effect on the pool size i.e maximum number
> of connections per route is staying the same (potentially for a very long
> time).
> E.g. let's use only two routes where the default maximum connections per
> route are 10 and the maximum total is 20. At some point, the maximum number
> of connections per route will be riched and the number of allocated
> connections (available + leased) will be 10. Now, let's say that we need to
> decrease the maximum number of connections per route to 5, for any reason.
> Doing that by calling either
> _PoolingHttpClientConnectionManager#setDefaultMaxPerRoute(int)_ or
> _PoolingHttpClientConnectionManager#setMaxPerRoute(HttpRoute, int)_ will not
> change the pool size and the maximum number of connections per route will
> stay the same (potentially for a very long time).
> This is the sample of the pool stats printed after changing +maxPerRoute+ to
> 5:
> {noformat}
> {somehost1:8983={"available":4,"leased":6,"max":5,"pending":0},
> somehost2:8080={"available":1,"leased":9,"max":5,"pending":2}}
> {somehost1:8983={"available":4,"leased":6,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":2}}
> {somehost1:8983={"available":5,"leased":5,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":4}}
> {somehost1:8983={"available":5,"leased":5,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":4}}
> {somehost1:8983={"available":7,"leased":3,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":5}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":5}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":7}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":8,"leased":2,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0},
> somehost2:8080={"available":0,"leased":10,"max":5,"pending":6}}
> {somehost1:8983={"available":9,"leased":1,"max":5,"pending":0},
> somehost2:8080={"available":1,"leased":9,"max":5,"pending":6}}} {noformat}
> As you can see, not only that the number of allocated connections per route
> is still 10, but also very often the number of leased connection is higher
> than 5. In my opinion, this is not what is expected and not really in favor
> of the principle of least surprise.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-484) check timeout could use TimeWheel algorithm
[ https://issues.apache.org/jira/browse/HTTPCORE-484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCORE-484. Fix Version/s: (was: Future) Resolution: Won't Fix Closing due to inactivity. Oleg > check timeout could use TimeWheel algorithm > --- > > Key: HTTPCORE-484 > URL: https://issues.apache.org/jira/browse/HTTPCORE-484 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore NIO >Affects Versions: 4.4.6 >Reporter: silver9886 >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > check time out every this.selectTimeout,and had to iterator all the > SelectionKeys. > it is not efficient nor accurate when their is a lot of SelectionKeys. > I suggest use the TimeWheel algorithm when check the channel time out just > as netty do. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2328.
---
Fix Version/s: (was: 5.3.2)
Resolution: Fixed
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2326) Support any Exception in RetryStrategy
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2326.
---
Fix Version/s: 5.4-alpha3
Resolution: Fixed
> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
> Fix For: 5.4-alpha3
>
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
> (it happens randomly that the proxy refuses the initial connection, but it
> usually works ok when retrying).
> I have tried extending
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
> but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of
> {{Exception }} or maybe {{TunnelRefusedException}} should be an
> {{IOException}}
>
> This is an example of a stacktrace that I would like to retry:
>
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy:
> HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)
[ https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCORE-766. Fix Version/s: 5.3-alpha3 Resolution: Fixed > Serve same content regardless of http authority (RequestHandlerRegistry) > > > Key: HTTPCORE-766 > URL: https://issues.apache.org/jira/browse/HTTPCORE-766 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2 >Reporter: Erik Wramner >Priority: Minor > Labels: easyfix > Fix For: 5.3-alpha3 > > Original Estimate: 4h > Time Spent: 40m > Remaining Estimate: 3h 20m > > I need to serve the same content on all interfaces and I don't know the IP > addresses and host names in advance. Think of it as a classic HTTP server > without virtual host support: all requests get to the same set of handlers. > Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses > one primary LookupRegistry for a canonical host name and localhost and a map > with other registries keyed by name. If a client connects with an unknown > name, no handler is found and the server responds with 421. > There are several possible solutions here: > # Modify RequestHandlerRegistry to use the primary registry if no other > registry is found. As I recall, that was what Apache did by default. Then it > is still possible to register new rules for virtual hosts, but the primary is > fallback when no name matches. This could also be configurable on/off. > # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual > hosts and always uses the primary. > # Don't fix this in the code, but make it easier for me to fix by making > getPatternMatcher protected instead of private. > In the two last cases we run into another problem: ServerBootstrap has > private members without getters, final methods that cannot be overridden and > creates RequestHandlerRegistry with new in a long create method. This is the > problem that was solved in HTTPCORE-570 for another class. It makes it hard > to extend the class, it must basically be rewritten. > I would propose opening up ServerBootstrap with getters or by making the > fields protected. I would also like to extract the RequestHandlerRegistry > creation to a protected method that can be overridden. Alternatively, we can > add a setter and allow the calling code to prepare an instance outside of the > builder. > I would be happy to make a pull request for these changes on Github. If so, > what direction would you like to take? > I can code around this by copying both classes with my changes, but that > feels really dirty. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-2329. --- Fix Version/s: 5.3.2 5.4-alpha3 Resolution: Duplicate > BasicHttpClientConnectionManager reuses closed connection objects > - > > Key: HTTPCLIENT-2329 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329 > Project: HttpComponents HttpClient > Issue Type: Bug >Affects Versions: 5.3.1 >Reporter: Teresa Tang >Priority: Major > Fix For: 5.3.2, 5.4-alpha3 > > Attachments: TestBasicConnectionManager.java > > > In the discardEndpoint method of InternalExecRuntime.java, the endpoint and > connection are closed. The manager releases the connection with a timevalue > of 0 ms. Because 0 is not considered positive, this leads to the expiration > being set to Long.MAX_VALUE. Upon the next connection request, the manager > will continue to use this unexpired connection object, even though it is > closed. > > The intention of the 0 ms timevalue was to have the connection expire > immediately and then be discarded so that the manager will create a new > connection for the next request. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851985#comment-17851985 ] Oleg Kalnichevski commented on HTTPCLIENT-2329: --- [~ttang] Alternatively get the source from GitHub [1], switch to 5.3.x branch and run `mvn clean install` to install the latest SNAPSHOT into your local Maven repository. Oleg [1] https://github.com/apache/httpcomponents-client/tree/5.3.x > BasicHttpClientConnectionManager reuses closed connection objects > - > > Key: HTTPCLIENT-2329 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329 > Project: HttpComponents HttpClient > Issue Type: Bug >Affects Versions: 5.3.1 >Reporter: Teresa Tang >Priority: Major > Attachments: TestBasicConnectionManager.java > > > In the discardEndpoint method of InternalExecRuntime.java, the endpoint and > connection are closed. The manager releases the connection with a timevalue > of 0 ms. Because 0 is not considered positive, this leads to the expiration > being set to Long.MAX_VALUE. Upon the next connection request, the manager > will continue to use this unexpired connection object, even though it is > closed. > > The intention of the 0 ms timevalue was to have the connection expire > immediately and then be discarded so that the manager will create a new > connection for the next request. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851984#comment-17851984
]
Oleg Kalnichevski commented on HTTPCLIENT-2328:
---
[~Zoe Wang] Ultimately the fix itself is fairly simple [1]. However it requires
quite a bit of refactoring in the blocking connection and TLS code [2]. For
one, it was no longer possible to use SSL socket auto-close mode and SSL
sockets now must be managed independently from their underling network socket.
This allows for greater control over TLS protocol at the price of greater
complexity.
These changes fix the problem for me locally. The use of
`MonitoringResponseOutOfOrderStrategy` is not necessary.
Please review / test locally/
Oleg
[1
]https://github.com/apache/httpcomponents-core/pull/468/commits/fc093c403ef4cbee9e0c6100fabe5a5e2ff73efc
[2]
[https://github.com/apache/httpcomponents-core/pull/468/commits/3277ffe85c76be0f0c6cefeec4d1b1412d5e256d]
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851808#comment-17851808 ] Oleg Kalnichevski commented on HTTPCLIENT-2329: --- [~ttang] This looks like a duplicate of HTTPCLIENT-2316. Could you please upgrade to the latest 5.3.2-SNAPSHOT and verify? Oleg > BasicHttpClientConnectionManager reuses closed connection objects > - > > Key: HTTPCLIENT-2329 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329 > Project: HttpComponents HttpClient > Issue Type: Bug >Affects Versions: 5.3.1 >Reporter: Teresa Tang >Priority: Major > Attachments: TestBasicConnectionManager.java > > > In the discardEndpoint method of InternalExecRuntime.java, the endpoint and > connection are closed. The manager releases the connection with a timevalue > of 0 ms. Because 0 is not considered positive, this leads to the expiration > being set to Long.MAX_VALUE. Upon the next connection request, the manager > will continue to use this unexpired connection object, even though it is > closed. > > The intention of the 0 ms timevalue was to have the connection expire > immediately and then be discarded so that the manager will create a new > connection for the next request. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851645#comment-17851645 ] Oleg Kalnichevski commented on HTTPCLIENT-2329: --- [~ttang] Could you please put together a test case demonstrating the defect like those in our test suite? https://github.com/apache/httpcomponents-client/blob/master/httpclient5-testing/src/test/java/org/apache/hc/client5/testing/sync/TestBasicConnectionManager.java Oleg > BasicHttpClientConnectionManager reuses closed connection objects > - > > Key: HTTPCLIENT-2329 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329 > Project: HttpComponents HttpClient > Issue Type: Bug >Affects Versions: 5.3.1 >Reporter: Teresa Tang >Priority: Major > > In the discardEndpoint method of InternalExecRuntime.java, the endpoint and > connection are closed. The manager releases the connection with a timevalue > of 0 ms. Because 0 is not considered positive, this leads to the expiration > being set to Long.MAX_VALUE. Upon the next connection request, the manager > will continue to use this unexpired connection object, even though it is > closed. > > The intention of the 0 ms timevalue was to have the connection expire > immediately and then be discarded so that the manager will create a new > connection for the next request. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2329) BasicHttpClientConnectionManager reuses closed connection objects
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851635#comment-17851635 ] Oleg Kalnichevski commented on HTTPCLIENT-2329: --- [~ttang] Please do specify the exact version of HttpClient you are using. Better yet make sure to upgrade to the latest GA release and re-test your code. Oleg > BasicHttpClientConnectionManager reuses closed connection objects > - > > Key: HTTPCLIENT-2329 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2329 > Project: HttpComponents HttpClient > Issue Type: Bug >Reporter: Teresa Tang >Priority: Major > > In the discardEndpoint method of InternalExecRuntime.java, the endpoint and > connection are closed. The manager releases the connection with a timevalue > of 0 ms. Because 0 is not considered positive, this leads to the expiration > being set to Long.MAX_VALUE. Upon the next connection request, the manager > will continue to use this unexpired connection object, even though it is > closed. > > The intention of the 0 ms timevalue was to have the connection expire > immediately and then be discarded so that the manager will create a new > connection for the next request. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851326#comment-17851326
]
Oleg Kalnichevski commented on HTTPCLIENT-2328:
---
[~Zoe Wang] I can reproduce the defect with the following code snippet using
HttpCore. Please note the fix will still require the user to explicitly
configure the client to use `MonitoringResponseOutOfOrderStrategy`.
{code:java}
SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial(Paths.get("keystore.jks"), "password".toCharArray())
.build();
HttpRequester requester = RequesterBootstrap.bootstrap()
.setSslContext(sslContext)
.setSslSetupHandler(sslParameters -> {
sslParameters.setProtocols(new String[]{TLS.V_1_3.id});
})
.setConnectionFactory(DefaultBHttpClientConnectionFactory.builder()
.responseOutOfOrderStrategy(MonitoringResponseOutOfOrderStrategy.INSTANCE)
.build())
.create();
HttpHost target = new HttpHost("https", "localhost", 8081);
// It needs to be a large file (128mb)
ClassicHttpRequest request = ClassicRequestBuilder.put("/")
.setHttpHost(target)
.setEntity(new PathEntity(Paths.get("output.dat"),
ContentType.APPLICATION_OCTET_STREAM))
.build();
HttpCoreContext context = HttpCoreContext.create();
try (ClassicHttpResponse response = requester.execute(target, request,
Timeout.ofMinutes(1), context)) {
EntityUtils.consume(response.getEntity());
}
{code}
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski updated HTTPCLIENT-2328:
--
Fix Version/s: 5.3.2
5.4-alpha3
Priority: Minor (was: Major)
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Minor
> Fix For: 5.3.2, 5.4-alpha3
>
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)
[ https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17851313#comment-17851313 ] Oleg Kalnichevski commented on HTTPCORE-766: [~erik.wramner] Feel free to review the actual pull request: [https://github.com/apache/httpcomponents-core/pull/467] If I hear no complaints or no feedback within 3 to 4 days I will merge the PR as is. Oleg > Serve same content regardless of http authority (RequestHandlerRegistry) > > > Key: HTTPCORE-766 > URL: https://issues.apache.org/jira/browse/HTTPCORE-766 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2 >Reporter: Erik Wramner >Priority: Minor > Labels: easyfix > Original Estimate: 4h > Time Spent: 10m > Remaining Estimate: 3h 50m > > I need to serve the same content on all interfaces and I don't know the IP > addresses and host names in advance. Think of it as a classic HTTP server > without virtual host support: all requests get to the same set of handlers. > Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses > one primary LookupRegistry for a canonical host name and localhost and a map > with other registries keyed by name. If a client connects with an unknown > name, no handler is found and the server responds with 421. > There are several possible solutions here: > # Modify RequestHandlerRegistry to use the primary registry if no other > registry is found. As I recall, that was what Apache did by default. Then it > is still possible to register new rules for virtual hosts, but the primary is > fallback when no name matches. This could also be configurable on/off. > # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual > hosts and always uses the primary. > # Don't fix this in the code, but make it easier for me to fix by making > getPatternMatcher protected instead of private. > In the two last cases we run into another problem: ServerBootstrap has > private members without getters, final methods that cannot be overridden and > creates RequestHandlerRegistry with new in a long create method. This is the > problem that was solved in HTTPCORE-570 for another class. It makes it hard > to extend the class, it must basically be rewritten. > I would propose opening up ServerBootstrap with getters or by making the > fields protected. I would also like to extract the RequestHandlerRegistry > creation to a protected method that can be overridden. Alternatively, we can > add a setter and allow the calling code to prepare an instance outside of the > builder. > I would be happy to make a pull request for these changes on Github. If so, > what direction would you like to take? > I can code around this by copying both classes with my changes, but that > feels really dirty. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850613#comment-17850613
]
Oleg Kalnichevski edited comment on HTTPCLIENT-2328 at 5/30/24 7:30 AM:
> I just don't see how it's violating HTTP spec.
[~Zoe Wang] HTTP is defined as a *request / response* based protocol. HTTP
servers are expected to respond with a response message to a request message,
If the server does not want the client to continue sending content of the
request message body it must still respond with an out of sequence response
message and not just half-close connections in the middle of message exchange.
That is how.
> The workaround we tried that seems to be working is do a
> "sock.getInputStream().read(new byte[0]);" before each write, which will
> check `conContext.isInboundClosed()` and raise SocketException if inbound is
> closed.
This is precisely what ResponseOutOfOrderStrategy does (which completely
decimates the i/o performance of the client, by the way). Feel free to use a
custom implementation or enhance the default one shipped with HttpClient.
Oleg
was (Author: olegk):
> I just don't see how it's violating HTTP spec.
[~Zoe Wang] HTTP is defined as a *request / response* based protocol. HTTP
servers are expected to respond with a response message to a request message,
If the server does not want the client to continue sending content of the
request message body it must still respond with an out of sequence response
message and not just half-close connections in the middle of message exchange.
That is how.
> The workaround we tried that seems to be working is do a
> "sock.getInputStream().read(new byte[0]);" before each write, which will
> check `conContext.isInboundClosed()` and raise SocketException if inbound is
> closed.
This is precisely what ResponseOutOfOrderStrategy does. Feel free to use a
custom implementation or enhance the default one shipped with HttpClient.
Oleg
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850613#comment-17850613
]
Oleg Kalnichevski commented on HTTPCLIENT-2328:
---
> I just don't see how it's violating HTTP spec.
[~Zoe Wang] HTTP is defined as a *request / response* based protocol. HTTP
servers are expected to respond with a response message to a request message,
If the server does not want the client to continue sending content of the
request message body it must still respond with an out of sequence response
message and not just half-close connections in the middle of message exchange.
That is how.
> The workaround we tried that seems to be working is do a
> "sock.getInputStream().read(new byte[0]);" before each write, which will
> check `conContext.isInboundClosed()` and raise SocketException if inbound is
> closed.
This is precisely what ResponseOutOfOrderStrategy does. Feel free to use a
custom implementation or enhance the default one shipped with HttpClient.
Oleg
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)
[
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850408#comment-17850408
]
Oleg Kalnichevski commented on HTTPCORE-766:
[~erik.wramner] I put together a new class {{RequestRouter}} [1] that I intend
to use as a replacement for {{RequestHandlerRegistry}}. The new class provides
new functionality that may be of interest to you: 1. it can use a custom
function to normalize or resolve service authorities. 2. when not authoritative
to handler a request it can delegate the request routing to a downstream
resolver. The basic use of the new class can be seen here [2]
You are welcome to review the change-set in my branch and propose API
improvements, better class / method names or ideas for additional functionality.
Oleg
[1]
https://github.com/ok2c/httpcomponents-core/blob/HTTPCORE-766/httpcore5/src/main/java/org/apache/hc/core5/http/impl/routing/RequestRouter.java
[2]
https://github.com/ok2c/httpcomponents-core/blob/HTTPCORE-766/httpcore5/src/test/java/org/apache/hc/core5/http/impl/routing/TestRequestRouter.java
> Serve same content regardless of http authority (RequestHandlerRegistry)
>
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
> Labels: easyfix
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP
> addresses and host names in advance. Think of it as a classic HTTP server
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses
> one primary LookupRegistry for a canonical host name and localhost and a map
> with other registries keyed by name. If a client connects with an unknown
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
> # Modify RequestHandlerRegistry to use the primary registry if no other
> registry is found. As I recall, that was what Apache did by default. Then it
> is still possible to register new rules for virtual hosts, but the primary is
> fallback when no name matches. This could also be configurable on/off.
> # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual
> hosts and always uses the primary.
> # Don't fix this in the code, but make it easier for me to fix by making
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has
> private members without getters, final methods that cannot be overridden and
> creates RequestHandlerRegistry with new in a long create method. This is the
> problem that was solved in HTTPCORE-570 for another class. It makes it hard
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the
> fields protected. I would also like to extract the RequestHandlerRegistry
> creation to a protected method that can be overridden. Alternatively, we can
> add a setter and allow the calling code to prepare an instance outside of the
> builder.
> I would be happy to make a pull request for these changes on Github. If so,
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that
> feels really dirty.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17850280#comment-17850280
]
Oleg Kalnichevski commented on HTTPCLIENT-2328:
---
> which is not technically compliant to the TCP 1.3 spec (outbound and inbound
> close_notify should be independent)
[~Zoe Wang] With all due respect what you are saying make no sense to me.
HTTP/1.1 is a request / response oriented protocol. It defines no semantic
meaning for half-closed connections. Moreover, the server half-closing its
endpoint while receiving a request message from the client would essentially be
in violation of the HTTP spec. The client can half-close its endpoint when
writing out requests in the pipeline mode. Even that, however, goes counter to
the spec that states that HTTP/1.1 are to be kept persistent.
Oleg
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849909#comment-17849909
]
Oleg Kalnichevski commented on HTTPCLIENT-2328:
---
[~Zoe Wang] What is it _specifically_ you expect me to do? As I tried to
explain in my previous response the classic i/o can either efficiently read
from the socket or efficiently write to the socket but not both. If need the
client to be able to efficiently react to asynchronous events you should
consider switching to the async transport provided by HttpClient.
Alternatively, enable `jdk.tls.acknowledgeCloseNotify`. I think this is the
reason it exists in the first place.
Oleg
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2328.
---
Resolution: Not A Problem
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2328) Request hangs if TLS 1.3 connection is half-closed
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1784#comment-1784
]
Oleg Kalnichevski commented on HTTPCLIENT-2328:
---
[~Zoe Wang] The Java classic i/o model has a well-known limitation: there is no
_efficient_ way of reading from and writing to the same socket at the same
time. While the process is busy writing out data it is unable to react to _any_
incoming events on the same socket. What one can do is solve the problem is to
perform an extra short read after each chunk of data written to the socket but
this has a substantial performance penalty.
We provide a ResponseOutOfOrderStrategy so one can configure HttpClient to
monitor connection for out-of-sequence events, but it is disabled (no op) by
default. It needs to be explicitly enabled.
{code:java}
final PoolingHttpClientConnectionManager cm =
PoolingHttpClientConnectionManagerBuilder.create()
.setConnectionFactory(ManagedHttpClientConnectionFactory.builder()
.responseOutOfOrderStrategy(MonitoringResponseOutOfOrderStrategy.INSTANCE)
.build())
.build();
try (CloseableHttpClient httpclient = HttpClients.custom()
.setConnectionManager(cm)
.build()) {
for (final URIScheme uriScheme : URIScheme.values()) {
final ClassicHttpRequest request = ClassicRequestBuilder.get()
.setHttpHost(new HttpHost(uriScheme.id, "httpbin.org"))
.setPath("/headers")
.build();
System.out.println("Executing request " + request);
httpclient.execute(request, response -> {
System.out.println("");
System.out.println(request + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
return null;
});
}
}
{code}
Oleg
> Request hangs if TLS 1.3 connection is half-closed
> ---
>
> Key: HTTPCLIENT-2328
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2328
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.5.14, 5.3.1
>Reporter: Zoe Wang
>Priority: Major
> Attachments: HalfCloseApache5Client.Java, HalfCloseServer.java,
> TlsHalfCloseApache4.java, keystore.jks
>
>
> If a server with TLS 1.3 support closes the connection during the request,
> more specifically, sending close_notify while the client is still writing to
> socket, the request will hang indefinitely. It's not an issue with TLS 1.2
> because it uses duplex-close policy. With TLS 1.3's half-closed connection
> policy, it seems Apache HTTP client is not able to detect connection closure
> properly. We are able to reproduce the issue with both 4.x and 5.x. I should
> note that HTTP URL connection does not have this issue.
> The workaround it to set `jdk.tls.acknowledgeCloseNotify` to true (see
> [https://bugs.openjdk.org/browse/JDK-8208526]), but that would require a lot
> of users to make changes on their side.
>
> Steps to repro:
> * Download the attached keystore file
> * Update ksPath in the server code HalfCloseServer.java to where you
> download the keystore
> * Run the server, the server will begin listening on {{localhost:8081}}
> * Create a random file of size 128MB and update client code "testFile" to
> where the file is.
> * Run the client, it should hang
> ** If System.setProperty("jdk.tls.acknowledgeCloseNotify", "true") is
> uncommented, it will not hang
> ** It also won’t hang if we we force TLS1.2
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)
[ https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847292#comment-17847292 ] Oleg Kalnichevski commented on HTTPCORE-766: [~erik.wramner] I may end up deprecating the whole handler registration logic in favor of a completely new request router. I am still looking into it. At any rate one does not need to register any handlers at all in which case all requests will get routed to the fallback mapper. > The workaround where authority is removed by an interceptor probably works, > but it feels a bit dangerous. Some other code may need the authority and go > south if it is null. The server does not need to remove request URI authority. it can replace it with a valid one it is authoritative for. Oleg > Serve same content regardless of http authority (RequestHandlerRegistry) > > > Key: HTTPCORE-766 > URL: https://issues.apache.org/jira/browse/HTTPCORE-766 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2 >Reporter: Erik Wramner >Priority: Minor > Labels: easyfix > Original Estimate: 4h > Remaining Estimate: 4h > > I need to serve the same content on all interfaces and I don't know the IP > addresses and host names in advance. Think of it as a classic HTTP server > without virtual host support: all requests get to the same set of handlers. > Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses > one primary LookupRegistry for a canonical host name and localhost and a map > with other registries keyed by name. If a client connects with an unknown > name, no handler is found and the server responds with 421. > There are several possible solutions here: > # Modify RequestHandlerRegistry to use the primary registry if no other > registry is found. As I recall, that was what Apache did by default. Then it > is still possible to register new rules for virtual hosts, but the primary is > fallback when no name matches. This could also be configurable on/off. > # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual > hosts and always uses the primary. > # Don't fix this in the code, but make it easier for me to fix by making > getPatternMatcher protected instead of private. > In the two last cases we run into another problem: ServerBootstrap has > private members without getters, final methods that cannot be overridden and > creates RequestHandlerRegistry with new in a long create method. This is the > problem that was solved in HTTPCORE-570 for another class. It makes it hard > to extend the class, it must basically be rewritten. > I would propose opening up ServerBootstrap with getters or by making the > fields protected. I would also like to extract the RequestHandlerRegistry > creation to a protected method that can be overridden. Alternatively, we can > add a setter and allow the calling code to prepare an instance outside of the > builder. > I would be happy to make a pull request for these changes on Github. If so, > what direction would you like to take? > I can code around this by copying both classes with my changes, but that > feels really dirty. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)
[
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847279#comment-17847279
]
Oleg Kalnichevski commented on HTTPCORE-766:
[~erik.wramner] I think what I am going to do is to make the existing server
bootstraps take `{color:#00}HttpRequestMapper` and use it as a fallback for
those requests whose authority cannot be resolved.
{color}
Oleg
`
> Serve same content regardless of http authority (RequestHandlerRegistry)
>
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
> Labels: easyfix
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP
> addresses and host names in advance. Think of it as a classic HTTP server
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses
> one primary LookupRegistry for a canonical host name and localhost and a map
> with other registries keyed by name. If a client connects with an unknown
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
> # Modify RequestHandlerRegistry to use the primary registry if no other
> registry is found. As I recall, that was what Apache did by default. Then it
> is still possible to register new rules for virtual hosts, but the primary is
> fallback when no name matches. This could also be configurable on/off.
> # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual
> hosts and always uses the primary.
> # Don't fix this in the code, but make it easier for me to fix by making
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has
> private members without getters, final methods that cannot be overridden and
> creates RequestHandlerRegistry with new in a long create method. This is the
> problem that was solved in HTTPCORE-570 for another class. It makes it hard
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the
> fields protected. I would also like to extract the RequestHandlerRegistry
> creation to a protected method that can be overridden. Alternatively, we can
> add a setter and allow the calling code to prepare an instance outside of the
> builder.
> I would be happy to make a pull request for these changes on Github. If so,
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that
> feels really dirty.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-766) Serve same content regardless of http authority (RequestHandlerRegistry)
[
https://issues.apache.org/jira/browse/HTTPCORE-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17847251#comment-17847251
]
Oleg Kalnichevski commented on HTTPCORE-766:
[~erik.wramner] One way to solve the problem once and for all is to drop or
rewrite the authority on all incoming requests
{code:java}
public class ClassicServerBootstrapTest {
private static final Timeout TIMEOUT = Timeout.ofMinutes(1);
@RegisterExtension
final HttpServerResource serverResource;
@RegisterExtension
final HttpRequesterResource clientResource;
public ClassicServerBootstrapTest() {
serverResource = new HttpServerResource(URIScheme.HTTP, bootstrap ->
bootstrap
.setSocketConfig(SocketConfig.custom()
.setSoTimeout(TIMEOUT)
.build())
.setHttpProcessor(HttpProcessors.customServer("test-server")
.add((HttpRequestInterceptor) (request, entity,
context) -> {
request.setAuthority(null);
})
.build()
)
.register("*", new EchoHandler()));
clientResource = new HttpRequesterResource(bootstrap -> bootstrap
.setSocketConfig(SocketConfig.custom()
.setSoTimeout(TIMEOUT)
.build()));
}
@Test
public void testStuff() throws Exception {
final HttpServer server = serverResource.start();
final HttpRequester requester = clientResource.start();
server.start();
final HttpHost target = new HttpHost("http", "localhost",
server.getLocalPort());
final HttpCoreContext context = HttpCoreContext.create();
final ClassicHttpRequest request =
ClassicRequestBuilder.get("http://somewhere.in.pampa/;)
.build();
try (final ClassicHttpResponse response = requester.execute(target,
request, TIMEOUT, context)) {
assertThat(response.getCode(),
CoreMatchers.equalTo(HttpStatus.SC_OK));
}
}
}
{code}
I will see if there is a reasonable way of plugging a custom replacement for
`RequestHandlerRegistry` but generally I do not thin that opening up
`ServerBootstrap` is a good idea.
Oleg
> Serve same content regardless of http authority (RequestHandlerRegistry)
>
>
> Key: HTTPCORE-766
> URL: https://issues.apache.org/jira/browse/HTTPCORE-766
> Project: HttpComponents HttpCore
> Issue Type: Improvement
> Components: HttpCore
>Affects Versions: 5.2.4, 5.3-alpha1, 5.3-alpha2
>Reporter: Erik Wramner
>Priority: Minor
> Labels: easyfix
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> I need to serve the same content on all interfaces and I don't know the IP
> addresses and host names in advance. Think of it as a classic HTTP server
> without virtual host support: all requests get to the same set of handlers.
> Unfortunately, ServerBootstrap installs RequestHandlerRegistry, which uses
> one primary LookupRegistry for a canonical host name and localhost and a map
> with other registries keyed by name. If a client connects with an unknown
> name, no handler is found and the server responds with 421.
> There are several possible solutions here:
> # Modify RequestHandlerRegistry to use the primary registry if no other
> registry is found. As I recall, that was what Apache did by default. Then it
> is still possible to register new rules for virtual hosts, but the primary is
> fallback when no name matches. This could also be configurable on/off.
> # Add new PrimaryOnlyRequestHandlerRegistry that has no support for virtual
> hosts and always uses the primary.
> # Don't fix this in the code, but make it easier for me to fix by making
> getPatternMatcher protected instead of private.
> In the two last cases we run into another problem: ServerBootstrap has
> private members without getters, final methods that cannot be overridden and
> creates RequestHandlerRegistry with new in a long create method. This is the
> problem that was solved in HTTPCORE-570 for another class. It makes it hard
> to extend the class, it must basically be rewritten.
> I would propose opening up ServerBootstrap with getters or by making the
> fields protected. I would also like to extract the RequestHandlerRegistry
> creation to a protected method that can be overridden. Alternatively, we can
> add a setter and allow the calling code to prepare an instance outside of the
> builder.
> I would be happy to make a pull request for these changes on Github. If so,
> what direction would you like to take?
> I can code around this by copying both classes with my changes, but that
> feels really dirty.
--
[jira] [Resolved] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2327.
---
Fix Version/s: 5.4-alpha3
Resolution: Fixed
> MemcachedHttpAsyncCacheStorage propagates CancellationException from
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
> Fix For: 5.4-alpha3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Spymemcached provides options to handle failure modes when an in-flight
> request to a Memcached instance fails (for example, when deploying to a
> Memcached server that is currently serving requests). One of the modes, and
> the most logical for our scenario in particular, is
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
> This, as you would expect, cancels any futures that are in-flight when a
> Memcached node becomes unreachable.
>
> The issue that we're having is that the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> (for restore linked here, as this is our issue, but also for other methods),
> doesn't handle the {{CancellationException}} gracefully, and instead
> propagates this Exception up to the caller. Since this is a cache for an HTTP
> client, I would expect the client to handle the cancellation as a "cache
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though
> I'm likely not thinking of some edge cases here) and proceed to make the
> request to the resource originally requested, since the cache is a mere
> optimization, and a failure in the cache should not impact the actual request
> to the resource we're targeting.
>
> We did POC this by extending the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> and overriding the {{restore}} method to explicitly handle
> {{{}CancellationException{}}}s and complete the future to treat it as a cache
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client
> itself, rather than us extending to override this behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846671#comment-17846671
]
Oleg Kalnichevski commented on HTTPCLIENT-2327:
---
[~jattisha] I agree. Fell free to submit a PR at GitHub with the change to that
effect.
Oleg
> MemcachedHttpAsyncCacheStorage propagates CancellationException from
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
>
> Spymemcached provides options to handle failure modes when an in-flight
> request to a Memcached instance fails (for example, when deploying to a
> Memcached server that is currently serving requests). One of the modes, and
> the most logical for our scenario in particular, is
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
> This, as you would expect, cancels any futures that are in-flight when a
> Memcached node becomes unreachable.
>
> The issue that we're having is that the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> (for restore linked here, as this is our issue, but also for other methods),
> doesn't handle the {{CancellationException}} gracefully, and instead
> propagates this Exception up to the caller. Since this is a cache for an HTTP
> client, I would expect the client to handle the cancellation as a "cache
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though
> I'm likely not thinking of some edge cases here) and proceed to make the
> request to the resource originally requested, since the cache is a mere
> optimization, and a failure in the cache should not impact the actual request
> to the resource we're targeting.
>
> We did POC this by extending the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> and overriding the {{restore}} method to explicitly handle
> {{{}CancellationException{}}}s and complete the future to treat it as a cache
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client
> itself, rather than us extending to override this behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846643#comment-17846643
]
Oleg Kalnichevski edited comment on HTTPCLIENT-2327 at 5/15/24 2:10 PM:
[~jattisha] From the design perspective the decision whether or not a cache
failure should be treated as a cache miss is to be taken by the protocol layer,
and _not_ by the storage backend. Unless I am missing something here the
protocol layer already treats `ResourceIOException` as a cache miss:
https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/BasicHttpAsyncCache.java#L160
What am I missing?
Oleg
was (Author: olegk):
[~jattisha] From the design perspective the decision whether or not a cache
failure should be treated as a cache miss is to be taken by the protocol layer,
and _not_ by the storage backend. Unless I am missing something here the
protocol layer already treats `ResourceIOException` as a cache miss:
[https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/BasicHttpAsyncCache.java#L130]
What am I missing?
Oleg
> MemcachedHttpAsyncCacheStorage propagates CancellationException from
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
>
> Spymemcached provides options to handle failure modes when an in-flight
> request to a Memcached instance fails (for example, when deploying to a
> Memcached server that is currently serving requests). One of the modes, and
> the most logical for our scenario in particular, is
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
> This, as you would expect, cancels any futures that are in-flight when a
> Memcached node becomes unreachable.
>
> The issue that we're having is that the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> (for restore linked here, as this is our issue, but also for other methods),
> doesn't handle the {{CancellationException}} gracefully, and instead
> propagates this Exception up to the caller. Since this is a cache for an HTTP
> client, I would expect the client to handle the cancellation as a "cache
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though
> I'm likely not thinking of some edge cases here) and proceed to make the
> request to the resource originally requested, since the cache is a mere
> optimization, and a failure in the cache should not impact the actual request
> to the resource we're targeting.
>
> We did POC this by extending the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> and overriding the {{restore}} method to explicitly handle
> {{{}CancellationException{}}}s and complete the future to treat it as a cache
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client
> itself, rather than us extending to override this behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2327) MemcachedHttpAsyncCacheStorage propagates CancellationException from cancelled cache requests to caller
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846643#comment-17846643
]
Oleg Kalnichevski commented on HTTPCLIENT-2327:
---
[~jattisha] From the design perspective the decision whether or not a cache
failure should be treated as a cache miss is to be taken by the protocol layer,
and _not_ by the storage backend. Unless I am missing something here the
protocol layer already treats `ResourceIOException` as a cache miss:
[https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/BasicHttpAsyncCache.java#L130]
What am I missing?
Oleg
> MemcachedHttpAsyncCacheStorage propagates CancellationException from
> cancelled cache requests to caller
> ---
>
> Key: HTTPCLIENT-2327
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2327
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCache
>Reporter: Jovan Attisha
>Priority: Major
>
> Spymemcached provides options to handle failure modes when an in-flight
> request to a Memcached instance fails (for example, when deploying to a
> Memcached server that is currently serving requests). One of the modes, and
> the most logical for our scenario in particular, is
> [Cancel|https://github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/FailureMode.java#L51-L54].
> This, as you would expect, cancels any futures that are in-flight when a
> Memcached node becomes unreachable.
>
> The issue that we're having is that the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> (for restore linked here, as this is our issue, but also for other methods),
> doesn't handle the {{CancellationException}} gracefully, and instead
> propagates this Exception up to the caller. Since this is a cache for an HTTP
> client, I would expect the client to handle the cancellation as a "cache
> miss" (I would almost expect this behavior for {_}all exceptions{_}, though
> I'm likely not thinking of some edge cases here) and proceed to make the
> request to the resource originally requested, since the cache is a mere
> optimization, and a failure in the cache should not impact the actual request
> to the resource we're targeting.
>
> We did POC this by extending the
> {{[MemcachedHttpAsyncCacheStorage|https://github.com/apache/httpcomponents-client/blob/master/httpclient5-cache/src/main/java/org/apache/hc/client5/http/impl/cache/memcached/MemcachedHttpAsyncCacheStorage.java#L184]}}
> and overriding the {{restore}} method to explicitly handle
> {{{}CancellationException{}}}s and complete the future to treat it as a cache
> miss, but before we proceed with this, we want to understand if:
> 1. The Apache HTTPClient agrees that this is the desired behavior
> 2. If so, would the Apache HTTP Client accept this behavior in the client
> itself, rather than us extending to override this behavior
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2326) Support any Exception in RetryStrategy
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838582#comment-17838582
]
Oleg Kalnichevski commented on HTTPCLIENT-2326:
---
> unnelRefusedException should be an IOException, or it should be encapsulated
> in an IOException
[~ofrias] No, it should not.
Please test this change-set:
[https://github.com/ok2c/httpcomponents-client/commit/888f9b1b5ae7f3ca02d429a9d1e2db59a62a412e]
Oleg
> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
> (it happens randomly that the proxy refuses the initial connection, but it
> usually works ok when retrying).
> I have tried extending
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
> but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of
> {{Exception }} or maybe {{TunnelRefusedException}} should be an
> {{IOException}}
>
> This is an example of a stacktrace that I would like to retry:
>
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy:
> HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2326) Support any Exception in RetryStrategy
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838559#comment-17838559
]
Oleg Kalnichevski commented on HTTPCLIENT-2326:
---
[~ofrias] My idea is to modify `TunnelRefusedExcepcion` to include the original
HTTP response message that `HttpRequestRetryStrategy#retryRequest` can act
upon. A custom `HttpRequestRetryStrategy` implementation in your code should
then be able to decide whether to retry the request based on the proxy response
rather than based on an I/O (or any other) exception.
Oleg
> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
> (it happens randomly that the proxy refuses the initial connection, but it
> usually works ok when retrying).
> I have tried extending
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
> but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of
> {{Exception }} or maybe {{TunnelRefusedException}} should be an
> {{IOException}}
>
> This is an example of a stacktrace that I would like to retry:
>
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy:
> HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2326) Support any Exception in RetryStrategy
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17838539#comment-17838539
]
Oleg Kalnichevski commented on HTTPCLIENT-2326:
---
[~ofrias] What I would like to propose is that `TunnelRefusedException ` would
get routed to the `HttpRequestRetryStrategy#retryRequest` that takes
`HttpResponse` as a parameter.
Oleg
> Support any Exception in RetryStrategy
> --
>
> Key: HTTPCLIENT-2326
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2326
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Oscar
>Priority: Minor
>
> It would be useful to modify {{HttpRequestRetryExec}} to allow
> {{HttpRequestRetryStrategy}} to support all kind of exceptions, not only
> those extending {{{}IOException{}}}.
> I would like HttpClient to retry requests when facing a
> [TunnelRefusedException|https://hc.apache.org/httpcomponents-client-5.1.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/TunnelRefusedException.html]
> (it happens randomly that the proxy refuses the initial connection, but it
> usually works ok when retrying).
> I have tried extending
> [DefaultHttpRequestRetryStrategy|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/DefaultHttpRequestRetryStrategy.java]
> but it can only capture exceptions inheriting from {{IOException.}}
> So, maybe {{HttpRequestRetryStrategy}} interface should support any kind of
> {{Exception }} or maybe {{TunnelRefusedException}} should be an
> {{IOException}}
>
> This is an example of a stacktrace that I would like to retry:
>
> {code:java}
> org.apache.hc.client5.http.ClientProtocolException: CONNECT refused by proxy:
> HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:173)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> [...]
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
> at java.base/java.lang.Thread.run(Thread.java:1583)
> Caused by: org.apache.hc.client5.http.impl.TunnelRefusedException: CONNECT
> refused by proxy: HTTP/1.1 500 Internal Server Error
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.createTunnelToTarget(ConnectExec.java:284)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:151)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCORE-759) Content-Length missing on null entity request content
[ https://issues.apache.org/jira/browse/HTTPCORE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski updated HTTPCORE-759: --- Fix Version/s: (was: 5.2.4) > Content-Length missing on null entity request content > - > > Key: HTTPCORE-759 > URL: https://issues.apache.org/jira/browse/HTTPCORE-759 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.2 >Reporter: Billy Jaime Beltran >Priority: Minor > Labels: easyfix > Fix For: 5.3-alpha1 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > When making a POST request with an empty body, the following difference in > behaviour was observed. > In HttpComponents 4.4.x, a null body causes a Content-Length zero header to > be added. > In HttpComponents 5.2.x, a null body causes no Content-Length headers being > added. > While upgrading an Apache Camel application, we noticed that this breaks > calls to an upstream IIS server which replies with 411 (Length Required). > That server expects either a Content-Length 0 or a Transfer-Encoding: chunked > header (with a chunk-size of zero) on requests that have a body semantic > (POST, PUT). > As the code currently stands written, if we manually set `Content-Length: 0` > a ProtocolException is thrown > [RequestContent.java:106|https://github.com/apache/httpcomponents-core/blob/e3c770b55602eb9e5a45dfe7c6a07a1adede2c95/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestContent.java#L106] > and if we call the constructor with overwrite, this header is removed anyway > without being replaced. > The previous behaviour (4.x) is documented > [RequestContent.java:102|https://github.com/apache/httpcomponents-core/blob/a5c117028b7c620974304636d52f06f172f1d08b/httpcore/src/main/java/org/apache/http/protocol/RequestContent.java#L102] > > A suggested fix: re-add the null check and set Content-Length to zero. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2325) Avoid adding "; charset=" for multipart/form-data requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2325.
---
Fix Version/s: 5.4-alpha3
Resolution: Fixed
> Avoid adding "; charset=" for multipart/form-data requests
> --
>
> Key: HTTPCLIENT-2325
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2325
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.14
>Reporter: Vladimir Sitnikov
>Priority: Major
> Fix For: 5.4-alpha3
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently, HttpClient adds {{; charset=}} to {{multipart/form-data}} which
> both
> 1) Breaks certain HTTP servers: see
> https://github.com/apache/jmeter/pull/6251,
> https://github.com/akka/akka-http/issues/338
> 2) Does not follow RFC 2046 and RFC 7578
> Even though including "charset" parameter is not explicitly forbidden by
> RFCs, there are known HTTP servers that can't parse such requests, so why
> generating the "charset=..." for multipart/form-data in the first place?
> See how RFC 7578 suggests setting the default charset:
> https://datatracker.ietf.org/doc/html/rfc7578#section-5.1.2
> They mention a {{_charset_}} field instead.
>
> Unfortunately, removal of {{multipartEntityBuilder.setCharset(charset);}} in
> the caller's code is not enough as HttpClient uses the supplied charset in
> {{HttpBrowserCompatibleMultipart}}.
> ---
> I suggest to avoid sending {{charset}} within {{multipart/form-data}} header,
> so it includes only {{boundary}} just like RFC 7578 samples.
> In other words, I suggest removing these lines:
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpmime/src/main/java/org/apache/http/entity/mime/MultipartEntityBuilder.java#L215-L217
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2325) Avoid adding "; charset=" for multipart/form-data requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827974#comment-17827974
]
Oleg Kalnichevski commented on HTTPCLIENT-2325:
---
> The contents of form data is US-ASCII (always)
No, it is not. It is defined by `Content-Transfer-Encoding` and can be binary.
Oleg
> Avoid adding "; charset=" for multipart/form-data requests
> --
>
> Key: HTTPCLIENT-2325
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2325
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.14
>Reporter: Vladimir Sitnikov
>Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Currently, HttpClient adds {{; charset=}} to {{multipart/form-data}} which
> both
> 1) Breaks certain HTTP servers: see
> https://github.com/apache/jmeter/pull/6251,
> https://github.com/akka/akka-http/issues/338
> 2) Does not follow RFC 2046 and RFC 7578
> Even though including "charset" parameter is not explicitly forbidden by
> RFCs, there are known HTTP servers that can't parse such requests, so why
> generating the "charset=..." for multipart/form-data in the first place?
> See how RFC 7578 suggests setting the default charset:
> https://datatracker.ietf.org/doc/html/rfc7578#section-5.1.2
> They mention a {{_charset_}} field instead.
>
> Unfortunately, removal of {{multipartEntityBuilder.setCharset(charset);}} in
> the caller's code is not enough as HttpClient uses the supplied charset in
> {{HttpBrowserCompatibleMultipart}}.
> ---
> I suggest to avoid sending {{charset}} within {{multipart/form-data}} header,
> so it includes only {{boundary}} just like RFC 7578 samples.
> In other words, I suggest removing these lines:
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpmime/src/main/java/org/apache/http/entity/mime/MultipartEntityBuilder.java#L215-L217
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2325) Avoid adding "; charset=" for multipart/form-data requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827905#comment-17827905
]
Oleg Kalnichevski commented on HTTPCLIENT-2325:
---
> What makes you think that?
[~michael-o] Common sense.
Oleg
> Avoid adding "; charset=" for multipart/form-data requests
> --
>
> Key: HTTPCLIENT-2325
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2325
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.14
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, HttpClient adds {{; charset=}} to {{multipart/form-data}} which
> both
> 1) Breaks certain HTTP servers: see
> https://github.com/apache/jmeter/pull/6251,
> https://github.com/akka/akka-http/issues/338
> 2) Does not follow RFC 2046 and RFC 7578
> Even though including "charset" parameter is not explicitly forbidden by
> RFCs, there are known HTTP servers that can't parse such requests, so why
> generating the "charset=..." for multipart/form-data in the first place?
> See how RFC 7578 suggests setting the default charset:
> https://datatracker.ietf.org/doc/html/rfc7578#section-5.1.2
> They mention a {{_charset_}} field instead.
>
> Unfortunately, removal of {{multipartEntityBuilder.setCharset(charset);}} in
> the caller's code is not enough as HttpClient uses the supplied charset in
> {{HttpBrowserCompatibleMultipart}}.
> ---
> I suggest to avoid sending {{charset}} within {{multipart/form-data}} header,
> so it includes only {{boundary}} just like RFC 7578 samples.
> In other words, I suggest removing these lines:
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpmime/src/main/java/org/apache/http/entity/mime/MultipartEntityBuilder.java#L215-L217
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2325) Avoid adding "; charset=" for multipart/form-data requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827898#comment-17827898
]
Oleg Kalnichevski commented on HTTPCLIENT-2325:
---
[~vladimirsitnikov] It may well be used as the default charset in case one has
not been explicitly set on any of the parts.
Oleg
> Avoid adding "; charset=" for multipart/form-data requests
> --
>
> Key: HTTPCLIENT-2325
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2325
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.14
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, HttpClient adds {{; charset=}} to {{multipart/form-data}} which
> both
> 1) Breaks certain HTTP servers: see
> https://github.com/apache/jmeter/pull/6251,
> https://github.com/akka/akka-http/issues/338
> 2) Does not follow RFC 2046 and RFC 7578
> Even though including "charset" parameter is not explicitly forbidden by
> RFCs, there are known HTTP servers that can't parse such requests, so why
> generating the "charset=..." for multipart/form-data in the first place?
> See how RFC 7578 suggests setting the default charset:
> https://datatracker.ietf.org/doc/html/rfc7578#section-5.1.2
> They mention a {{_charset_}} field instead.
>
> Unfortunately, removal of {{multipartEntityBuilder.setCharset(charset);}} in
> the caller's code is not enough as HttpClient uses the supplied charset in
> {{HttpBrowserCompatibleMultipart}}.
> ---
> I suggest to avoid sending {{charset}} within {{multipart/form-data}} header,
> so it includes only {{boundary}} just like RFC 7578 samples.
> In other words, I suggest removing these lines:
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpmime/src/main/java/org/apache/http/entity/mime/MultipartEntityBuilder.java#L215-L217
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2325) Avoid adding "; charset=" for multipart/form-data requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827890#comment-17827890
]
Oleg Kalnichevski commented on HTTPCLIENT-2325:
---
> Oleg, as they say in [https://github.com/akka/akka-http/issues/338], IIS 8.5
> does not understand what multipart with charset means.
[~vladimirsitnikov] How is this our problem, and not that of Microsoft?
> We can't immediately bump HTC in Apache JMeter from 4.5.x to 5.x for backward
> compatibility reasons, so it would be nice if there was 4.x release though.
I can back-port the fix to 4.5 but I still cannot promise there would be an
official release from the 4.5.x branch any time soon, if ever. Please consider
upgrading to 5.x
Oleg
> Avoid adding "; charset=" for multipart/form-data requests
> --
>
> Key: HTTPCLIENT-2325
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2325
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.14
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, HttpClient adds {{; charset=}} to {{multipart/form-data}} which
> both
> 1) Breaks certain HTTP servers: see
> https://github.com/apache/jmeter/pull/6251,
> https://github.com/akka/akka-http/issues/338
> 2) Does not follow RFC 2046 and RFC 7578
> Even though including "charset" parameter is not explicitly forbidden by
> RFCs, there are known HTTP servers that can't parse such requests, so why
> generating the "charset=..." for multipart/form-data in the first place?
> See how RFC 7578 suggests setting the default charset:
> https://datatracker.ietf.org/doc/html/rfc7578#section-5.1.2
> They mention a {{_charset_}} field instead.
>
> Unfortunately, removal of {{multipartEntityBuilder.setCharset(charset);}} in
> the caller's code is not enough as HttpClient uses the supplied charset in
> {{HttpBrowserCompatibleMultipart}}.
> ---
> I suggest to avoid sending {{charset}} within {{multipart/form-data}} header,
> so it includes only {{boundary}} just like RFC 7578 samples.
> In other words, I suggest removing these lines:
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpmime/src/main/java/org/apache/http/entity/mime/MultipartEntityBuilder.java#L215-L217
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2325) Avoid adding "; charset=" for multipart/form-data requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827877#comment-17827877
]
Oleg Kalnichevski commented on HTTPCLIENT-2325:
---
[~vladimirsitnikov] Any half-decent HTTP protocol handler must be able to parse
`Content-Type` header correctly, pick out parameters that it understands and
ignore those that it does not. In my option this is non-issue.
Having said that you are welcome to raise a PR against master (5.4) with the
changes you are proposing. Please note there will be no changes to 4.5.x beyond
critical fixes.
Oleg
> Avoid adding "; charset=" for multipart/form-data requests
> --
>
> Key: HTTPCLIENT-2325
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2325
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.14
>Reporter: Vladimir Sitnikov
>Priority: Major
>
> Currently, HttpClient adds {{; charset=}} to {{multipart/form-data}} which
> both
> 1) Breaks certain HTTP servers: see
> https://github.com/apache/jmeter/pull/6251,
> https://github.com/akka/akka-http/issues/338
> 2) Does not follow RFC 2046 and RFC 7578
> Even though including "charset" parameter is not explicitly forbidden by
> RFCs, there are known HTTP servers that can't parse such requests, so why
> generating the "charset=..." for multipart/form-data in the first place?
> See how RFC 7578 suggests setting the default charset:
> https://datatracker.ietf.org/doc/html/rfc7578#section-5.1.2
> They mention a {{_charset_}} field instead.
>
> Unfortunately, removal of {{multipartEntityBuilder.setCharset(charset);}} in
> the caller's code is not enough as HttpClient uses the supplied charset in
> {{HttpBrowserCompatibleMultipart}}.
> ---
> I suggest to avoid sending {{charset}} within {{multipart/form-data}} header,
> so it includes only {{boundary}} just like RFC 7578 samples.
> In other words, I suggest removing these lines:
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpmime/src/main/java/org/apache/http/entity/mime/MultipartEntityBuilder.java#L215-L217
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Assigned] (HTTPCORE-754) StrictConnPool unable to acquire lock under multi threaded application causing DeadlineTimeoutException exception being thrown
[ https://issues.apache.org/jira/browse/HTTPCORE-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski reassigned HTTPCORE-754: -- Assignee: Oleg Kalnichevski > StrictConnPool unable to acquire lock under multi threaded application > causing DeadlineTimeoutException exception being thrown > -- > > Key: HTTPCORE-754 > URL: https://issues.apache.org/jira/browse/HTTPCORE-754 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore >Affects Versions: 5.2, 5.2.2 >Reporter: Sreenivas S >Assignee: Oleg Kalnichevski >Priority: Major > Fix For: 5.2.3, 5.3-alpha1 > > Attachments: image-2023-06-30-18-27-24-293.png > > > Under load when two different threads try to get a connection one thread will > be able to get the connection lease and another thread will fail with > misleading *DeadlineTimeoutException* exception. > Exception > [20/Jun/2023:16:08:03-133] [ERROR] - Deadline: > +292278994-08-17T07:12:55.807+, 9223370349577492674 MILLISECONDS overdue > org.apache.hc.core5.util.DeadlineTimeoutException: Deadline: > +292278994-08-17T07:12:55.807+, 9223370349577492674 MILLISECONDS overdue > at > org.apache.hc.core5.util.DeadlineTimeoutException.from(DeadlineTimeoutException.java:49) > ~[httpcore5-5.2.2.jar:5.2.2] > at org.apache.hc.core5.pool.StrictConnPool.lease(StrictConnPool.java:217) > ~[httpcore5-5.2.2.jar:5.2.2] > at > org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager$3.(PoolingAsyncClientConnectionManager.java:271) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager.lease(PoolingAsyncClientConnectionManager.java:266) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.InternalHttpAsyncExecRuntime.acquireEndpoint(InternalHttpAsyncExecRuntime.java:105) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncConnectExec.execute(AsyncConnectExec.java:141) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncExecChainElement.execute(AsyncExecChainElement.java:54) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncProtocolExec.internalExecute(AsyncProtocolExec.java:207) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncProtocolExec.execute(AsyncProtocolExec.java:172) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncExecChainElement.execute(AsyncExecChainElement.java:54) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncHttpRequestRetryExec.internalExecute(AsyncHttpRequestRetryExec.java:97) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncHttpRequestRetryExec.execute(AsyncHttpRequestRetryExec.java:184) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.AsyncExecChainElement.execute(AsyncExecChainElement.java:54) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.InternalAbstractHttpAsyncClient.executeImmediate(InternalAbstractHttpAsyncClient.java:347) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.InternalAbstractHttpAsyncClient.lambda$doExecute$0(InternalAbstractHttpAsyncClient.java:205) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.core5.http.nio.support.BasicRequestProducer.sendRequest(BasicRequestProducer.java:93) > ~[httpcore5-5.2.2.jar:5.2.2] > at > org.apache.hc.client5.http.impl.async.InternalAbstractHttpAsyncClient.doExecute(InternalAbstractHttpAsyncClient.java:178) > ~[httpclient5-5.2.1.jar:5.2.1] > at > org.apache.hc.client5.http.impl.async.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:97) > ~[httpclient5-5.2.1.jar:5.2.1] > > DeadlineTimeoutException error is misleading in this case and actually > HTTPCore it is unable to acquire lock (From the exception provided earlier > Line 217 of StrictConnPool.java). Timeout of 0 means don't wait as per java > docs of ReentrantLock. So in my case it looks like already one thread > acquired lock and another thread now trying to acquire lock and as TimeOut > value is 0, that thread is returning back and causing that exception to be > thrown. As per HTTPClient java docs a timeout of 0 to > setConnectionRequestTimeout means infinite timeout. > > *Use below code to generate misleading error* > Timeout requestTimeout = Timeout.ZERO_MILLISECONDS; > final Deadline deadline = Deadline.calculate(requestTimeout); > System.out.println(DeadlineTimeoutException.from(deadline)); > -- This
[jira] [Resolved] (HTTPCLIENT-2324) Apache HttpClient 5 blocks for double the duration set by requestTimeout when using TLS
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2324.
---
Resolution: Invalid
> Apache HttpClient 5 blocks for double the duration set by requestTimeout when
> using TLS
> ---
>
> Key: HTTPCLIENT-2324
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2324
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: OpenJDK Runtime Environment Corretto-17.0.8.7.1
>Reporter: Tim Davidson
>Priority: Minor
>
> I have an issue in my application using Spring's RestTemplate configured with
> Apache HttpClient 5 as the underlying http client. My application is calling
> a downstream legacy system, and while that system is "down" for maintenance,
> it still accepts connections (TLS handshake and all), but never returns a
> response.
> The problem I'm having is when I set the request timeout at 8 seconds on the
> Apache client, requests sent to this downstream service are blocking for
> double that timeout setting (16 seconds).
> I have this isolated to the Apache HttpClient, and have reproduced this
> behavior in a unit test using self-signed certs. Since the code necessary to
> reproduce is too long to post here, I have a repository with the test
> reproducing the issue here:
> [apache-httpclient-socket-timeout-test|https://github.com/tndavidson/apache-httpclient-socket-timeout-test]
> You can see in the following logs that the read timeout does happen after the
> configured 8 seconds, but it is blocking for an additional 8 seconds to close
> the connection. This doesn't happen using HTTP, only HTTPS.
> {code:java}
> 14:45:21.587 [main] DEBUG org.apache.hc.client5.http.wire -- http-outgoing-1
> >> "[\r][\n]"
> 14:45:29.589 [main] DEBUG org.apache.hc.client5.http.wire -- http-outgoing-1
> << "[read] I/O error: Read timed out"
> 14:45:29.589 [main] DEBUG o.a.h.c.h.i.i.DefaultManagedHttpClientConnection --
> http-outgoing-1 Close connection
> 14:45:37.591 [main] DEBUG o.a.h.c.h.i.c.InternalHttpClient --
> InternalConnectionEndpoint-7eb6b6b6 endpoint closed {code}
> I've included two additional tests that show this issue is 1) not present
> when not configured for TLS, and 2) not present when configuring the same TLS
> configuration using the JDK's native HttpClient.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2324) Apache HttpClient 5 blocks for double the duration set by requestTimeout when using TLS
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825776#comment-17825776
]
Oleg Kalnichevski commented on HTTPCLIENT-2324:
---
[~tndavid...@gmail.com] Again, socket timeout is _NOT_ a deadline. It ensures
that the connection socket cannot be inactive longer that the value of timeout
at {_}a point of time{_}. In other words, if the socket timeout is N and the
server sends a packet every N - 1, the connection _never_ times out. Never. At
all.
Do you know how many packets get exchanged between the local and the remote
endpoint during the TLS handshake? How many DNS queries get performed? Are
there any automatic retries or redirects?
Basically your expectations are wrong. Socket timeout value of 16 seconds does
not guarantee a socket timeout exception immediately after 16 seconds of
execution time. What it guarantees is that a single i/o operation cannot take
longer than 16 seconds.
Oleg
> Apache HttpClient 5 blocks for double the duration set by requestTimeout when
> using TLS
> ---
>
> Key: HTTPCLIENT-2324
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2324
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: OpenJDK Runtime Environment Corretto-17.0.8.7.1
>Reporter: Tim Davidson
>Priority: Minor
>
> I have an issue in my application using Spring's RestTemplate configured with
> Apache HttpClient 5 as the underlying http client. My application is calling
> a downstream legacy system, and while that system is "down" for maintenance,
> it still accepts connections (TLS handshake and all), but never returns a
> response.
> The problem I'm having is when I set the request timeout at 8 seconds on the
> Apache client, requests sent to this downstream service are blocking for
> double that timeout setting (16 seconds).
> I have this isolated to the Apache HttpClient, and have reproduced this
> behavior in a unit test using self-signed certs. Since the code necessary to
> reproduce is too long to post here, I have a repository with the test
> reproducing the issue here:
> [apache-httpclient-socket-timeout-test|https://github.com/tndavidson/apache-httpclient-socket-timeout-test]
> You can see in the following logs that the read timeout does happen after the
> configured 8 seconds, but it is blocking for an additional 8 seconds to close
> the connection. This doesn't happen using HTTP, only HTTPS.
> {code:java}
> 14:45:21.587 [main] DEBUG org.apache.hc.client5.http.wire -- http-outgoing-1
> >> "[\r][\n]"
> 14:45:29.589 [main] DEBUG org.apache.hc.client5.http.wire -- http-outgoing-1
> << "[read] I/O error: Read timed out"
> 14:45:29.589 [main] DEBUG o.a.h.c.h.i.i.DefaultManagedHttpClientConnection --
> http-outgoing-1 Close connection
> 14:45:37.591 [main] DEBUG o.a.h.c.h.i.c.InternalHttpClient --
> InternalConnectionEndpoint-7eb6b6b6 endpoint closed {code}
> I've included two additional tests that show this issue is 1) not present
> when not configured for TLS, and 2) not present when configuring the same TLS
> configuration using the JDK's native HttpClient.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2324) Apache HttpClient 5 blocks for double the duration set by requestTimeout when using TLS
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17825714#comment-17825714
]
Oleg Kalnichevski commented on HTTPCLIENT-2324:
---
[~tndavid...@gmail.com] Are you sure your expectations are correct? A socket
timeout represents the maximum period of inactivity between two consecutive i/o
operations, not a deadline in absolute terms.
Oleg
> Apache HttpClient 5 blocks for double the duration set by requestTimeout when
> using TLS
> ---
>
> Key: HTTPCLIENT-2324
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2324
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: OpenJDK Runtime Environment Corretto-17.0.8.7.1
>Reporter: Tim Davidson
>Priority: Minor
>
> I have an issue in my application using Spring's RestTemplate configured with
> Apache HttpClient 5 as the underlying http client. My application is calling
> a downstream legacy system, and while that system is "down" for maintenance,
> it still accepts connections (TLS handshake and all), but never returns a
> response.
> The problem I'm having is when I set the request timeout at 8 seconds on the
> Apache client, requests sent to this downstream service are blocking for
> double that timeout setting (16 seconds).
> I have this isolated to the Apache HttpClient, and have reproduced this
> behavior in a unit test using self-signed certs. Since the code necessary to
> reproduce is too long to post here, I have a repository with the test
> reproducing the issue here:
> [apache-httpclient-socket-timeout-test|https://github.com/tndavidson/apache-httpclient-socket-timeout-test]
> You can see in the following logs that the read timeout does happen after the
> configured 8 seconds, but it is blocking for an additional 8 seconds to close
> the connection. This doesn't happen using HTTP, only HTTPS.
> {code:java}
> 14:45:21.587 [main] DEBUG org.apache.hc.client5.http.wire -- http-outgoing-1
> >> "[\r][\n]"
> 14:45:29.589 [main] DEBUG org.apache.hc.client5.http.wire -- http-outgoing-1
> << "[read] I/O error: Read timed out"
> 14:45:29.589 [main] DEBUG o.a.h.c.h.i.i.DefaultManagedHttpClientConnection --
> http-outgoing-1 Close connection
> 14:45:37.591 [main] DEBUG o.a.h.c.h.i.c.InternalHttpClient --
> InternalConnectionEndpoint-7eb6b6b6 endpoint closed {code}
> I've included two additional tests that show this issue is 1) not present
> when not configured for TLS, and 2) not present when configuring the same TLS
> configuration using the JDK's native HttpClient.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822084#comment-17822084
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
[~winstonwaite] HttpClient version 4 and version 5 have been intentionally
designed so that HttpClient instances need to be created very rarely, most once
per application lifetime and then repeatedly re-used. Pretty much _each_ and
_every_ aspect of the library can be overwritten either on a per route
(connection manager level) or on a per request (HttpContext level) with a very
few notable exceptions. Do note though that not every parameter makes sense at
the request level. For instance, connect timeout is applicable to the route
level only. It is not applicable at the request level. Socket timeout _can_ be
overwritten at the request level via HttpContext.
What you are doing is _wasteful_ and _inefficient_ per design and a tweak to
the library is not going to change that.
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes. Spring boot 2.7.x.
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822041#comment-17822041
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
> PoolingHttpClientConnectionManagerBuilder does not exist in 4.5.x.
[~winstonwaite] So what? The principle is exactly the same.
We do not make any changes to 4.x other than critical fixes. Consider
upgrading. Or whatever.
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes. Spring boot 2.7.x.
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822032#comment-17822032
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
> Do you think this should be called once per application run...
Yes, i do.
> or multiple times? I am saying you need to call it multiple times in order to
> alter the connection pool settings.
You are wrong.
> Are you saying you can call that once and never call it again while still
> altering the connection pool used?
Yes, I am.
> Besides this particular point... is it truly the ideal design for the apache
> http client to resort to the classloader to determine a default user agent?
> Can this not be calculated once? Or in less resource-intensive way? Are we
> losing something by rethinking this code?
Given this is supposed to used be very infrequently I see no point caching the
default user agent value. Again, there is nothing stopping you from caching and
re-suing it in your code.
Truly, I do not understand what else I need to say
{code:java}
final PoolingHttpClientConnectionManager cm =
PoolingHttpClientConnectionManagerBuilder.create()
.build();
try (CloseableHttpClient httpclient = HttpClients.custom()
.setConnectionManager(cm)
.build()) {
for (final URIScheme uriScheme : URIScheme.values()) {
final ClassicHttpRequest request = ClassicRequestBuilder.get()
.setHttpHost(new HttpHost(uriScheme.id, "httpbin.org"))
.setPath("/headers")
.build();
System.out.println("Executing request " + request);
// Apply config dynamically
cm.setConnectionConfigResolver(route -> {
// Use different settings for all secure (TLS) connections
if (route.isSecure()) {
return ConnectionConfig.custom()
.setConnectTimeout(Timeout.ofMinutes(2))
.setSocketTimeout(Timeout.ofMinutes(2))
.setValidateAfterInactivity(TimeValue.ofMinutes(1))
.setTimeToLive(TimeValue.ofHours(1))
.build();
} else {
return ConnectionConfig.custom()
.setConnectTimeout(Timeout.ofMinutes(1))
.setSocketTimeout(Timeout.ofMinutes(1))
.setValidateAfterInactivity(TimeValue.ofSeconds(15))
.setTimeToLive(TimeValue.ofMinutes(15))
.build();
}
});
cm.setTlsConfigResolver(host -> {
// Use different settings for specific hosts
if (host.getSchemeName().equalsIgnoreCase("httpbin.org")) {
return TlsConfig.custom()
.setSupportedProtocols(TLS.V_1_3)
.setHandshakeTimeout(Timeout.ofSeconds(10))
.build();
} else {
return TlsConfig.DEFAULT;
}
});
httpclient.execute(request, response -> {
System.out.println("");
System.out.println(request + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
return null;
});
}
}
{code}
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes. Spring boot 2.7.x.
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822019#comment-17822019
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
[~winstonwaite] There is _nothing_ stopping you from re-configuring the
connection manager at runtime without needing to touch the clients that make
use of that connection manager. If you are not willing to see it I cannot help
you. And one can set different connection and TLS settings on a per route basis
without having to have different connection pools. If you are not willing to
see it I cannot help you.
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2321) Clarify the default connection manager being used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822011#comment-17822011
]
Oleg Kalnichevski commented on HTTPCLIENT-2321:
---
> Your referenced code. The configured connection pool is used to instantiate
> the CloseableHttpClient. How can you reuse ClosableHttpClient if you want to
> alter the connection pool config on the fly?
[~kwin] I disagree. I one needs a specific behavior one must explicitly
configure it. In the next release the default might change (and we as project
developers need to have freedom to do so).
Oleg
> Clarify the default connection manager being used
> -
>
> Key: HTTPCLIENT-2321
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2321
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: Documentation
>Affects Versions: 5.4-alpha1
>Reporter: Konrad Windszus
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Neither from reading
> https://hc.apache.org/httpcomponents-client-4.5.x/current/tutorial/html/connmgmt.html
> nor from the javadocs at either
> #
> https://hc.apache.org/httpcomponents-client-5.3.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.html,
> #
> https://hc.apache.org/httpcomponents-client-5.3.x/current/httpclient5/apidocs/org/apache/hc/client5/http/io/HttpClientConnectionManager.html
> or
> #
> https://hc.apache.org/httpcomponents-client-5.3.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/io/PoolingHttpClientConnectionManager.html
> it is clear that the {{PoolingHttpClientConnectionManager}} is used by
> default if no other connection manager is set explicitly
> (https://github.com/apache/httpcomponents-client/blob/f2b9a374d817ee630a2efe706546ba4f1e5a5ac4/httpclient5/src/main/java/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.java#L754).
> I was always assuming that due to the [KISS
> principle|https://en.wikipedia.org/wiki/KISS_principle] the one named
> {{BasicHttpClientConnectionManager}} would be the default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822009#comment-17822009
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
> How can you reuse ClosableHttpClient if you want to alter the connection pool
> config on the fly?
[~winstonwaite] What is that stops you?
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17822003#comment-17822003
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
> you often want to have different timeouts for the connection... or use
> different configurations for connection pools.
[~winstonwaite] All This is doable without touching the HttpClient instance
through HttpContext and connection pool config. There is no excuse to not
re-using HttpClient instances.
https://github.com/apache/httpcomponents-client/blob/5.3.x/httpclient5/src/test/java/org/apache/hc/client5/http/examples/ClientConnectionConfig.java
https://github.com/apache/httpcomponents-client/blob/5.3.x/httpclient5/src/test/java/org/apache/hc/client5/http/examples/ClientConfiguration.java
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2323.
---
Resolution: Not A Problem
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12, 5.3.1
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821982#comment-17821982
]
Oleg Kalnichevski commented on HTTPCLIENT-2323:
---
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
[~winstonwaite] No, it is not. This is _very_ wrong and is the cause of the
problem you are seeing. One should not need more than a single HttpClient
instance per distinct HTTP service. Please do consider re-using HttpClient
instances.
Oleg
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2321) Clarify the default connection manager being used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821860#comment-17821860
]
Oleg Kalnichevski commented on HTTPCLIENT-2321:
---
[~kwin] `{{{}BasicHttpClientConnectionManager{}}}` was created in those dark
days when people used to put HttpClient inside a stateless session EJB and
needed an implementation backed by a single connection. It was _never_ meant to
be the default implementation.
Generally HttpClient builders should be able to pick the best implementation as
they deem fit without making it a part of the public contract if those
implementations have not been explicitly set by the consumer.
I am not in favor of this change.
Oleg
> Clarify the default connection manager being used
> -
>
> Key: HTTPCLIENT-2321
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2321
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: Documentation
>Affects Versions: 5.4-alpha1
>Reporter: Konrad Windszus
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Neither from reading
> https://hc.apache.org/httpcomponents-client-4.5.x/current/tutorial/html/connmgmt.html
> nor from the javadocs at either
> #
> https://hc.apache.org/httpcomponents-client-5.3.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.html,
> #
> https://hc.apache.org/httpcomponents-client-5.3.x/current/httpclient5/apidocs/org/apache/hc/client5/http/io/HttpClientConnectionManager.html
> or
> #
> https://hc.apache.org/httpcomponents-client-5.3.x/current/httpclient5/apidocs/org/apache/hc/client5/http/impl/io/PoolingHttpClientConnectionManager.html
> it is clear that the {{PoolingHttpClientConnectionManager}} is used by
> default if no other connection manager is set explicitly
> (https://github.com/apache/httpcomponents-client/blob/f2b9a374d817ee630a2efe706546ba4f1e5a5ac4/httpclient5/src/main/java/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.java#L754).
> I was always assuming that due to the [KISS
> principle|https://en.wikipedia.org/wiki/KISS_principle] the one named
> {{BasicHttpClientConnectionManager}} would be the default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2320) Support IPv6 proxy exceptions in OSGiHttpRoutePlanner
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821073#comment-17821073
]
Oleg Kalnichevski commented on HTTPCLIENT-2320:
---
[~kwin] I am sorry but I will have close this one as WONTFIX
# We as a project no longer support OSGi
# HC 4.x gets no fixes other than for major protocol violations or critical
security issues
Having said that you are welcome to contribute a fix. It will be reviewed and
merged. What I cannot promise that will ever make into an official release.
Oleg
> Support IPv6 proxy exceptions in OSGiHttpRoutePlanner
> -
>
> Key: HTTPCLIENT-2320
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2320
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.15
>Reporter: Konrad Windszus
>Priority: Minor
>
> The code in
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpclient-osgi/src/main/java/org/apache/http/osgi/impl/OSGiHttpRoutePlanner.java#L67
> can only correctly deal with host name, domain name or IPv4 address
> exceptions. This is due to the fact that the inner class {{NetworkAddress}}
> only parses IPv4 format address specs (and only stores the address internally
> in an int = 32 bit)
> (https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpclient-osgi/src/main/java/org/apache/http/osgi/impl/OSGiHttpRoutePlanner.java#L154-L177)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2320) Support IPv6 proxy exceptions in OSGiHttpRoutePlanner
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2320.
---
Resolution: Won't Fix
> Support IPv6 proxy exceptions in OSGiHttpRoutePlanner
> -
>
> Key: HTTPCLIENT-2320
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2320
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.15
>Reporter: Konrad Windszus
>Priority: Minor
>
> The code in
> https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpclient-osgi/src/main/java/org/apache/http/osgi/impl/OSGiHttpRoutePlanner.java#L67
> can only correctly deal with host name, domain name or IPv4 address
> exceptions. This is due to the fact that the inner class {{NetworkAddress}}
> only parses IPv4 format address specs (and only stores the address internally
> in an int = 32 bit)
> (https://github.com/apache/httpcomponents-client/blob/54900db4653d7f207477e6ee40135b88e9bcf832/httpclient-osgi/src/main/java/org/apache/http/osgi/impl/OSGiHttpRoutePlanner.java#L154-L177)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2318) Implement HttpClientConnectionManager#isShutdown
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820783#comment-17820783 ] Oleg Kalnichevski commented on HTTPCLIENT-2318: --- [~edmund.ham] No, it will not. Consider migrating to HC 5. Oleg > Implement HttpClientConnectionManager#isShutdown > > > Key: HTTPCLIENT-2318 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2318 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Reporter: Edmund Ham >Assignee: Arturo Bernal >Priority: Major > Fix For: 5.4-alpha2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hi all, > We have an internal library that has core logics and lots of microservices > are using it. Within the internal library, we have "HttpClientPool" where we > are caching different HttpClient based on keys and microservices are using > them. Recently, we've been seeing some of OOM errors on microservices when > using HttpClients from our "HttpClientPool". > From https://issues.apache.org/jira/browse/HTTPCLIENT-2039 and > https://issues.apache.org/jira/browse/HTTPCLIENT-1924 we understand that it > is the intention to shutdown the connection pool in case of Java Errors. I'm > now wondering if it's possible to have new API, > "HttpClientConnectionManager#isShutdown" so that we as a library can identify > if the connection pool is closed or not, before giving out to our clients? > Currently there seems to be no way to identify it, unless we actually try to > make an outbound call and get "IllegalStateException: Connection pool shut > down" exception message, which makes impossible for us to do because we as a > library don't want to make additional HTTP client call every time when our > client requests for the HttpClient instance. > Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2318) Implement HttpClientConnectionManager#isShutdown
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-2318. --- Fix Version/s: 5.4-alpha2 Resolution: Fixed > Implement HttpClientConnectionManager#isShutdown > > > Key: HTTPCLIENT-2318 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2318 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Reporter: Edmund Ham >Assignee: Arturo Bernal >Priority: Major > Fix For: 5.4-alpha2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hi all, > We have an internal library that has core logics and lots of microservices > are using it. Within the internal library, we have "HttpClientPool" where we > are caching different HttpClient based on keys and microservices are using > them. Recently, we've been seeing some of OOM errors on microservices when > using HttpClients from our "HttpClientPool". > From https://issues.apache.org/jira/browse/HTTPCLIENT-2039 and > https://issues.apache.org/jira/browse/HTTPCLIENT-1924 we understand that it > is the intention to shutdown the connection pool in case of Java Errors. I'm > now wondering if it's possible to have new API, > "HttpClientConnectionManager#isShutdown" so that we as a library can identify > if the connection pool is closed or not, before giving out to our clients? > Currently there seems to be no way to identify it, unless we actually try to > make an outbound call and get "IllegalStateException: Connection pool shut > down" exception message, which makes impossible for us to do because we as a > library don't want to make additional HTTP client call every time when our > client requests for the HttpClient instance. > Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2319) DateUtils.parseStandardDate() throws exception instead of returning null
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17820017#comment-17820017
]
Oleg Kalnichevski commented on HTTPCLIENT-2319:
---
[~bwatermann] You are welcome to propose a fix by raising a PR at GitHub.
Oleg
> DateUtils.parseStandardDate() throws exception instead of returning null
>
>
> Key: HTTPCLIENT-2319
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2319
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Bernd Watermann
>Priority: Minor
>
> The following line of code
> {code:java}
> org.apache.hc.client5.http.utils.DateUtils.parseStandardDate("Thu Feb 22
> 17:20:18 2024");{code}
> throws an exception with the following stack trace:
> {noformat}
> java.time.DateTimeException: Unable to obtain Instant from TemporalAccessor:
> {},ISO resolved to 2024-02-22T17:20:18 of type java.time.format.Parsed
> at java.base/java.time.Instant.from(Instant.java:380)
> at
> org.apache.hc.client5.http.utils.DateUtils.parseDate(DateUtils.java:165)
> at
> org.apache.hc.client5.http.utils.DateUtils.parseStandardDate(DateUtils.java:183){noformat}
> The documentation for parseStandardDate(), however, says:
> ??Returns: the parsed instant or null if input could not be parsed??
> Throwing an exception does not seem to conform to that spec.
> Note that I don't care (at present) if the given string should actually be
> parseable or not. The primary issue is error handling and the fact that any
> error should lead to a null result and not to an exception being thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-765) Add content-length header when entity is null
[ https://issues.apache.org/jira/browse/HTTPCORE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCORE-765. Resolution: Invalid > Add content-length header when entity is null > - > > Key: HTTPCORE-765 > URL: https://issues.apache.org/jira/browse/HTTPCORE-765 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore >Affects Versions: 5.3-alpha2 >Reporter: Vishal Bihani >Priority: Minor > > When entity details is null, by default it should add a content-length header > with value 0. Not adding that results in 411 (Content Length required) error. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-765) Add content-length header when entity is null
[ https://issues.apache.org/jira/browse/HTTPCORE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818277#comment-17818277 ] Oleg Kalnichevski commented on HTTPCORE-765: > I am working on this [~_vishalbihani] Please do not. This has been discussed multiple times and this idea has been rejected multiple times. Please either add an empty entity to the request if you want the protocol handler to generate a `content-length` header automatically or use a custom request interceptor to add whatever custom logic you deem appropriate for your application. Oleg > Add content-length header when entity is null > - > > Key: HTTPCORE-765 > URL: https://issues.apache.org/jira/browse/HTTPCORE-765 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore >Affects Versions: 5.3-alpha2 >Reporter: Vishal Bihani >Priority: Minor > > When entity details is null, by default it should add a content-length header > with value 0. Not adding that results in 411 (Content Length required) error. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2318) Implement HttpClientConnectionManager#isShutdown
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17817968#comment-17817968
]
Oleg Kalnichevski commented on HTTPCLIENT-2318:
---
[~edmund.ham] Would it be sufficient if we added this method to the
{{PoolingHttpClientConnectionManager}} class, and not the
{{HttpClientConnectionManager}} interface?
Oleg
> Implement HttpClientConnectionManager#isShutdown
>
>
> Key: HTTPCLIENT-2318
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2318
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Reporter: Edmund Ham
>Priority: Major
>
> Hi all,
> We have an internal library that has core logics and lots of microservices
> are using it. Within the internal library, we have "HttpClientPool" where we
> are caching different HttpClient based on keys and microservices are using
> them. Recently, we've been seeing some of OOM errors on microservices when
> using HttpClients from our "HttpClientPool".
> From https://issues.apache.org/jira/browse/HTTPCLIENT-2039 and
> https://issues.apache.org/jira/browse/HTTPCLIENT-1924 we understand that it
> is the intention to shutdown the connection pool in case of Java Errors. I'm
> now wondering if it's possible to have new API,
> "HttpClientConnectionManager#isShutdown" so that we as a library can identify
> if the connection pool is closed or not, before giving out to our clients?
> Currently there seems to be no way to identify it, unless we actually try to
> make an outbound call and get "IllegalStateException: Connection pool shut
> down" exception message, which makes impossible for us to do because we as a
> library don't want to make additional HTTP client call every time when our
> client requests for the HttpClient instance.
> Thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-764) Apparent infinite loop in SSLIOSession
[ https://issues.apache.org/jira/browse/HTTPCORE-764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCORE-764. Resolution: Won't Fix [~bjhham] HttpAsyncClient 4.1.x and HttpCore NIO 4.4.x are end-of-life now. We no longer support or provide any fixes for those versions. Please consider upgrading to HttpCore 5.2 / HttpClient 5.3. As far as I know OpenSearch supports HC 5. Having said that, if you provide a fix for this defect I will review and commit it to the 4.4.x branch. Oleg > Apparent infinite loop in SSLIOSession > -- > > Key: HTTPCORE-764 > URL: https://issues.apache.org/jira/browse/HTTPCORE-764 > Project: HttpComponents HttpCore > Issue Type: Bug > Components: HttpCore NIO >Affects Versions: 4.4.15 >Reporter: Bruce Hamilton >Priority: Major > > Using the apache http async client 4.1.5, we are making calls to the Open AI > server-side streaming endpoint. After some period of making requests (an > hour or so), the service will hit 100% CPU and invariably the client will > invariably be stuck in the same spot in the SSLIOSession class, trying to > complete the handshake. > We believe the server is dropping connections during the TLS handshake, which > manifests as connection timeouts in other clients. This seems to happen for > both TLS 1.2 and 1.3, and I can see there's been some closely related issues > in the past regarding this loop. > > Here is the stack trace from the thread dump: > "Ktor-client-apache (server): llm-" #53 daemon prio=5 os_prio=0 > cpu=20726438.17ms elapsed=62011.90s tid=0x7f7f28074010 nid=0x41 runnable > [0x7f7e934fd000] > java.lang.Thread.State: RUNNABLE > at > sun.security.ssl.SSLEngineImpl.wrap(java.base@17.0.8/SSLEngineImpl.java:123) > at javax.net.ssl.SSLEngine.wrap(java.base@17.0.8/SSLEngine.java:524) > at > org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:270) > at > org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:316) > at > org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:547) > - locked <0xf7e699b0> (a > org.apache.http.nio.reactor.ssl.SSLIOSession) > at > org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120) > at > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) > at > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) > at > org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) > at > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) > at java.lang.Thread.run(java.base@17.0.8/Thread.java:833) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2317) The license in the httpclient5 artifact on Maven central is not correct.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17814784#comment-17814784 ] Oleg Kalnichevski commented on HTTPCLIENT-2317: --- [~ggregory] [~michael-o] Our project LICENSE.txt [1] includes all the requisite 3rd party licenses. This is a matter of making Maven include the damn thing. Oleg [1] https://github.com/apache/httpcomponents-client/blob/master/LICENSE.txt > The license in the httpclient5 artifact on Maven central is not correct. > > > Key: HTTPCLIENT-2317 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2317 > Project: HttpComponents HttpClient > Issue Type: Bug >Affects Versions: 4.5.14, 5.3.1, 5.4-alpha1 >Reporter: Jan Lahoda >Priority: Minor > > Looking at the 'httpclient5' artifact on Maven central, like: > [https://repo1.maven.org/maven2/org/apache/httpcomponents/client5/httpclient5/5.3.1/httpclient5-5.3.1.jar] > [https://repo1.maven.org/maven2/org/apache/httpcomponents/client5/httpclient5/5.4-alpha1/httpclient5-5.4-alpha1.jar] > these contain 'mozilla/public-suffix-list.txt', which is under MPL 2.0, but > the license file embedded in these artifacts (META-INF/LICENSE) does not > contain the MPL 2.0. > > The binary file downloadable from Apache download server has the correct > license file: > [https://dlcdn.apache.org//httpcomponents/httpclient/binary/httpcomponents-client-5.3.1-bin.tar.gz] > > It would be nice if the artifacts on Maven central could have the correct > content of the license file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2317) The license in the httpclient5 artifact on Maven central is not correct.
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski updated HTTPCLIENT-2317: -- Priority: Minor (was: Major) [~jlahoda] You are welcome to propose a fix as a PR at GitHub. Oleg > The license in the httpclient5 artifact on Maven central is not correct. > > > Key: HTTPCLIENT-2317 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2317 > Project: HttpComponents HttpClient > Issue Type: Bug >Affects Versions: 5.3.1, 5.4-alpha1 >Reporter: Jan Lahoda >Priority: Minor > > Looking at the 'httpclient5' artifact on Maven central, like: > [https://repo1.maven.org/maven2/org/apache/httpcomponents/client5/httpclient5/5.3.1/httpclient5-5.3.1.jar] > [https://repo1.maven.org/maven2/org/apache/httpcomponents/client5/httpclient5/5.4-alpha1/httpclient5-5.4-alpha1.jar] > these contain 'mozilla/public-suffix-list.txt', which is under MPL 2.0, but > the license file embedded in these artifacts (META-INF/LICENSE) does not > contain the MPL 2.0. > > The binary file downloadable from Apache download server has the correct > license file: > [https://dlcdn.apache.org//httpcomponents/httpclient/binary/httpcomponents-client-5.3.1-bin.tar.gz] > > It would be nice if the artifacts on Maven central could have the correct > content of the license file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2151) Optionally use JSSE inbuilt endpoint identification
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2151.
---
Resolution: Fixed
> Optionally use JSSE inbuilt endpoint identification
> ---
>
> Key: HTTPCLIENT-2151
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2151
> Project: HttpComponents HttpClient
> Issue Type: Wish
> Components: HttpClient (classic)
>Reporter: Peter Dettman
>Priority: Minor
> Fix For: 5.4-alpha2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> This issue brings to the fore that part of HTTPCLIENT-2134 which is actually
> within the scope of HttpClient: the possibility of using JSSE inbuilt
> endpoint identification ("HTTPS") instead of performing those checks
> externally (typically in
> _org.apache.hc.client5.http.ssl.DefaultHostnameVerifier_).
> Java 1.7 added the method
> SSLParameters.setEndpointIdentificationAlgorithm(String). Calling this method
> with the argument "HTTPS" (case-insensitive) will result in the endpoint
> identification checks from RFC 2818 being automatically done by JSSE during
> the handshake (specifically when the X509TrustManager validates the server
> certificate). Note that the setting can also be interrogated via
> SSLParameters.getEndpointIdentificationAlgorithm.
> This has some minor advantages relative to performing the checks externally:
> * The check is done earlier in the handshake, reducing the cost of failed
> connections. (Technically this might still be achieved anyway by wrapping the
> X509TrustManager and applying a HostnameVerifier at that stage).
> * Per my comments in HTTPCLIENT-2134, in some nuanced situations SunJSSE will
> be willing to resume a session only if it knows for sure that endpoint
> identification is being used (which it can't know for an arbitrary
> HostnameVerifier).
> * SunJSSE's HTTPS endpoint identification has special (less-restrictive)
> rules for wildcard characters when the trust anchor is a certificate from the
> 'cacerts' file (what it calls a "public root CA").
> There may be other implications that I am unaware of, but I think this
> establishes that some users may prefer (or even need) to delegate endpoint
> identification to JSSE in this way. They presumably will not then want a
> duplicate check in DefaultHostnameVerifier.
> I am not yet that familiar with HttpClient, but I assume a user could already
> provide a custom SSLSocketFactory which enables JSSE endpoint identification
> on sockets before returning them, and then separately configure a no-op
> HostnameVerifier. I consider that to be rather fragile and it's easy to see
> mistakes leading to no endpoint identification being performed.
> I am proposing that HttpClient support this via a simpler configuration
> mechanism. To loosely outline some basic options:
> IGNORE: don't even check whether JSSE endpoint identification was enabled
> (i.e. assume it wasn't).
> AWARE: check at runtime whether JSSE endpoint identification was "HTTPS" for
> a given socket connection.
> DISABLE: remove any existing setting for JSSE endpoint identification (set it
> to "").
> FORCE: overwrite any existing setting for JSSE endpoint identification (set
> it to "HTTPS").
> So we would consider some value that came to us via the SSLSocketFactory's
> default socket setup, and whether to ignore it, respect it, or force it
> on/off. DefaultHostnameVerifier can then be set to possibly skip its own
> endpoint checks when appropriate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2151) Optionally use JSSE inbuilt endpoint identification
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813105#comment-17813105
]
Oleg Kalnichevski commented on HTTPCLIENT-2151:
---
[~peter.dettman] Feel free to review the PR with the proposed changes:
[https://github.com/apache/httpcomponents-client/pull/545]
Oleg
> Optionally use JSSE inbuilt endpoint identification
> ---
>
> Key: HTTPCLIENT-2151
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2151
> Project: HttpComponents HttpClient
> Issue Type: Wish
> Components: HttpClient (classic)
>Reporter: Peter Dettman
>Priority: Minor
> Fix For: 5.4-alpha2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This issue brings to the fore that part of HTTPCLIENT-2134 which is actually
> within the scope of HttpClient: the possibility of using JSSE inbuilt
> endpoint identification ("HTTPS") instead of performing those checks
> externally (typically in
> _org.apache.hc.client5.http.ssl.DefaultHostnameVerifier_).
> Java 1.7 added the method
> SSLParameters.setEndpointIdentificationAlgorithm(String). Calling this method
> with the argument "HTTPS" (case-insensitive) will result in the endpoint
> identification checks from RFC 2818 being automatically done by JSSE during
> the handshake (specifically when the X509TrustManager validates the server
> certificate). Note that the setting can also be interrogated via
> SSLParameters.getEndpointIdentificationAlgorithm.
> This has some minor advantages relative to performing the checks externally:
> * The check is done earlier in the handshake, reducing the cost of failed
> connections. (Technically this might still be achieved anyway by wrapping the
> X509TrustManager and applying a HostnameVerifier at that stage).
> * Per my comments in HTTPCLIENT-2134, in some nuanced situations SunJSSE will
> be willing to resume a session only if it knows for sure that endpoint
> identification is being used (which it can't know for an arbitrary
> HostnameVerifier).
> * SunJSSE's HTTPS endpoint identification has special (less-restrictive)
> rules for wildcard characters when the trust anchor is a certificate from the
> 'cacerts' file (what it calls a "public root CA").
> There may be other implications that I am unaware of, but I think this
> establishes that some users may prefer (or even need) to delegate endpoint
> identification to JSSE in this way. They presumably will not then want a
> duplicate check in DefaultHostnameVerifier.
> I am not yet that familiar with HttpClient, but I assume a user could already
> provide a custom SSLSocketFactory which enables JSSE endpoint identification
> on sockets before returning them, and then separately configure a no-op
> HostnameVerifier. I consider that to be rather fragile and it's easy to see
> mistakes leading to no endpoint identification being performed.
> I am proposing that HttpClient support this via a simpler configuration
> mechanism. To loosely outline some basic options:
> IGNORE: don't even check whether JSSE endpoint identification was enabled
> (i.e. assume it wasn't).
> AWARE: check at runtime whether JSSE endpoint identification was "HTTPS" for
> a given socket connection.
> DISABLE: remove any existing setting for JSSE endpoint identification (set it
> to "").
> FORCE: overwrite any existing setting for JSSE endpoint identification (set
> it to "HTTPS").
> So we would consider some value that came to us via the SSLSocketFactory's
> default socket setup, and whether to ignore it, respect it, or force it
> on/off. DefaultHostnameVerifier can then be set to possibly skip its own
> endpoint checks when appropriate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-750) manage connections with upgradeable security
[ https://issues.apache.org/jira/browse/HTTPCLIENT-750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-750. -- Fix Version/s: 5.4-alpha2 (was: Future) Resolution: Fixed Fixed by HTTPCLIENT-751 Oleg > manage connections with upgradeable security > > > Key: HTTPCLIENT-750 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-750 > Project: HttpComponents HttpClient > Issue Type: New Feature > Components: HttpClient (classic) >Reporter: Roland Weber >Priority: Major > Fix For: 5.4-alpha2 > > > RFC 2817 (http://tools.ietf.org/html/rfc2817) specifies not only tunnelling > of TLS/SSL via proxies, but also upgrading an existing HTTP connection to > TLS/SSL. The latter is not commonly used for communication with traditional > HTTP servers, but part of other protocols like IPP that are based on HTTP. > The current connection management code assumes that a route planner can > determine in advance whether a connection will be secure or not. A connection > manager will not reuse an existing unsecure connection if a secure connection > is requested. It probably also doesn't consider a returned connection secure > if it wasn't requested for a secure route. > One way to improve the situation is to give HttpRoute a trinary security > flag, with values plain/secure for the current usage scenario and a new value > upgradeable for the new scenario. The two scenarios won't mix, but that is > probably not required. > We have to make sure that upgrade to security of an existing plain HTTP > connection is correctly tracked and either respected or suitably ignored by > the connection manager. If the security flag of the route is 'upgradeable', > mixing of scenarios is not required, and the actual security level can be > obtained from the connection itself, it is probably safe to let the > connection manager ignore it. > cheers, > Roland -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2316.
---
Fix Version/s: 5.3.2
5.4-alpha2
Resolution: Fixed
Committed to 5.3.x and master
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
> Fix For: 5.3.2, 5.4-alpha2
>
> Attachments: log-output-38d8d9b.txt, log-output-8266a9c.txt,
> log-output.txt
>
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Resolved] (HTTPCLIENT-2070) BasicAuthCache not to use object serialization
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-2070. --- Resolution: Fixed > BasicAuthCache not to use object serialization > -- > > Key: HTTPCLIENT-2070 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2070 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Alex Dettinger >Priority: Major > Labels: volunteers-wanted > Fix For: 5.4-alpha2 > > Time Spent: 50m > Remaining Estimate: 0h > > Today, BasicAuthCache is using object serialization in [put/get > methods|https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/impl/auth/BasicAuthCache.java#L91..L93]. > However, object serialization is not supported with [quarkus native > mode|https://quarkus.io/guides/building-native-image] and folk ends up > removing this part. > Folks ends up removing serialization by [native code > override|https://github.com/quarkusio/quarkus/blob/master/extensions/elasticsearch-rest-client/runtime/src/main/java/io/quarkus/elasticsearch/restclient/runtime/graal/Substitute_RestClient.java#L93..L114]. > Would it be possible to have httpclient finding an implementation of put/get > not using object serialization ? > What is the consequence of removing the serialization/deserialization ? > Thread safety issue ? persistence issue ? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812628#comment-17812628
]
Oleg Kalnichevski commented on HTTPCLIENT-2316:
---
[~windy] Try again.
[https://github.com/apache/httpcomponents-client/compare/5.3.x..HTTPCLIENT-2316]
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
> Attachments: log-output-38d8d9b.txt, log-output-8266a9c.txt,
> log-output.txt
>
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Commented] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812573#comment-17812573
]
Oleg Kalnichevski commented on HTTPCLIENT-2316:
---
[~windy] Please try again
[https://github.com/apache/httpcomponents-client/compare/5.3.x..HTTPCLIENT-2316]
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
> Attachments: log-output-38d8d9b.txt, log-output-8266a9c.txt,
> log-output.txt
>
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Commented] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812417#comment-17812417
]
Oleg Kalnichevski commented on HTTPCLIENT-2316:
---
[~windy] Give it another try:
[https://github.com/apache/httpcomponents-client/compare/5.3.x..HTTPCLIENT-2316]
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
> Attachments: log-output-38d8d9b.txt, log-output.txt
>
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Commented] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812390#comment-17812390
]
Oleg Kalnichevski commented on HTTPCLIENT-2316:
---
[~windy] Please review and test the proposed fix:
[https://github.com/apache/httpcomponents-client/compare/5.3.x..HTTPCLIENT-2316]
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
> Attachments: log-output.txt
>
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Commented] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812375#comment-17812375
]
Oleg Kalnichevski commented on HTTPCLIENT-2316:
---
[~windy] Please replace the basic connection manager with the pooling one and
see if that resolves the problem.
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
> Attachments: log-output.txt
>
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Commented] (HTTPCLIENT-2316) ConnectionShutdownException when connection is closed for 401 response
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17812338#comment-17812338
]
Oleg Kalnichevski commented on HTTPCLIENT-2316:
---
[~windy] Please at the very least post the _complete_ wire / context log of the
session or I will have to close this tickets as CANT_REPRODUCE.
Oleg
> ConnectionShutdownException when connection is closed for 401 response
> --
>
> Key: HTTPCLIENT-2316
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2316
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
> Environment: Ubuntu Linux, Java 17.0.10, connecting to an Apache Web
> Server
>Reporter: Stephan Windmüller
>Priority: Major
>
> When a connection is used for multiple requests, the {{Keep-Alive}} header
> has a {{max}} value that is reduced until it reaches 0, resulting in a
> {{Connection: closed}} header. If that happens while a 401 response is
> handled, the following code is executed:
> {code:java}
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.markConnectionNonReusable(InternalExecRuntime.java:233)
> at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:142)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> at
> org.apache.hc.client5.http.fluent.Request.internalExecute(Request.java:201)
> at org.apache.hc.client5.http.fluent.Executor.execute(Executor.java:244)
> {code}
> After that,
> {{org.apache.hc.client5.http.impl.classic.InternalExecRuntime#disconnectEndpoint}}
> is called. Any further attempt to use that connection results in this
> stacktrace:
> {code:java}
> org.apache.hc.client5.http.impl.ConnectionShutdownException
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.getConnection(BasicHttpClientConnectionManager.java:502)
> at
> org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager$InternalConnectionEndpoint.isConnected(BasicHttpClientConnectionManager.java:519)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.isEndpointConnected(InternalExecRuntime.java:149)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:128)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> at
>
[jira] [Resolved] (HTTPCLIENT-751) ProtocolSocketFactory to upgrade an unsecured connection to secured via TLS
[ https://issues.apache.org/jira/browse/HTTPCLIENT-751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-751. -- Resolution: Fixed > ProtocolSocketFactory to upgrade an unsecured connection to secured via TLS > --- > > Key: HTTPCLIENT-751 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-751 > Project: HttpComponents HttpClient > Issue Type: New Feature > Environment: Coded for Java 1.4.2, but interested parties should be > able to make it work for less than that. >Reporter: Stephen O'Neil Butler >Priority: Minor > Fix For: 5.4-alpha2 > > Attachments: TLSProtocolSocketFactory.java > > Time Spent: 50m > Remaining Estimate: 0h > > HTTP allows a client to upgrade its connection from an unsecured one to a > secured one via the Upgrade header. This protocol component implements a > mandatory upgrade. From RFC2817, section 3.2: > >> OPTIONS * HTTP/1.1 > >> Host: example.bank.com > >> Upgrade: TLS/1.0 > >> Connection: Upgrade > >> > << HTTP/1.1 101 Switching Protocols > << Upgrade: TLS/1.0, HTTP/1.1 > << Connection: Upgrade > << > -- send and complete the TLS handshake > -- server answers the original OPTIONS request > << HTTP/1.1 200 OK > << Date: Sun, 17 Feb 2008 17:23:35 GMT > << Server: Apache > << Content-Length: 0 > << Content-Type: text/plain > << > My reason for developing this protocol socket factory was to support secure > IPP over the standard IPP port. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-2151) Optionally use JSSE inbuilt endpoint identification
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810436#comment-17810436
]
Oleg Kalnichevski edited comment on HTTPCLIENT-2151 at 1/24/24 3:17 PM:
[~peter.dettman] I need to find (or put together) a reasonable test case to
proceed. I had no luck with HTTPCLIENT-2134
Oleg
was (Author: olegk):
[~peter.dettman] I need to find (or put together) a reasonable test to proceed.
I had no luck with HTTPCLIENT-2134
Oleg
> Optionally use JSSE inbuilt endpoint identification
> ---
>
> Key: HTTPCLIENT-2151
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2151
> Project: HttpComponents HttpClient
> Issue Type: Wish
> Components: HttpClient (classic)
>Reporter: Peter Dettman
>Priority: Minor
> Fix For: 5.4-alpha2
>
>
> This issue brings to the fore that part of HTTPCLIENT-2134 which is actually
> within the scope of HttpClient: the possibility of using JSSE inbuilt
> endpoint identification ("HTTPS") instead of performing those checks
> externally (typically in
> _org.apache.hc.client5.http.ssl.DefaultHostnameVerifier_).
> Java 1.7 added the method
> SSLParameters.setEndpointIdentificationAlgorithm(String). Calling this method
> with the argument "HTTPS" (case-insensitive) will result in the endpoint
> identification checks from RFC 2818 being automatically done by JSSE during
> the handshake (specifically when the X509TrustManager validates the server
> certificate). Note that the setting can also be interrogated via
> SSLParameters.getEndpointIdentificationAlgorithm.
> This has some minor advantages relative to performing the checks externally:
> * The check is done earlier in the handshake, reducing the cost of failed
> connections. (Technically this might still be achieved anyway by wrapping the
> X509TrustManager and applying a HostnameVerifier at that stage).
> * Per my comments in HTTPCLIENT-2134, in some nuanced situations SunJSSE will
> be willing to resume a session only if it knows for sure that endpoint
> identification is being used (which it can't know for an arbitrary
> HostnameVerifier).
> * SunJSSE's HTTPS endpoint identification has special (less-restrictive)
> rules for wildcard characters when the trust anchor is a certificate from the
> 'cacerts' file (what it calls a "public root CA").
> There may be other implications that I am unaware of, but I think this
> establishes that some users may prefer (or even need) to delegate endpoint
> identification to JSSE in this way. They presumably will not then want a
> duplicate check in DefaultHostnameVerifier.
> I am not yet that familiar with HttpClient, but I assume a user could already
> provide a custom SSLSocketFactory which enables JSSE endpoint identification
> on sockets before returning them, and then separately configure a no-op
> HostnameVerifier. I consider that to be rather fragile and it's easy to see
> mistakes leading to no endpoint identification being performed.
> I am proposing that HttpClient support this via a simpler configuration
> mechanism. To loosely outline some basic options:
> IGNORE: don't even check whether JSSE endpoint identification was enabled
> (i.e. assume it wasn't).
> AWARE: check at runtime whether JSSE endpoint identification was "HTTPS" for
> a given socket connection.
> DISABLE: remove any existing setting for JSSE endpoint identification (set it
> to "").
> FORCE: overwrite any existing setting for JSSE endpoint identification (set
> it to "HTTPS").
> So we would consider some value that came to us via the SSLSocketFactory's
> default socket setup, and whether to ignore it, respect it, or force it
> on/off. DefaultHostnameVerifier can then be set to possibly skip its own
> endpoint checks when appropriate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2151) Optionally use JSSE inbuilt endpoint identification
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810436#comment-17810436
]
Oleg Kalnichevski commented on HTTPCLIENT-2151:
---
[~peter.dettman] I need to find (or put together) a reasonable test to proceed.
I had no luck with HTTPCLIENT-2134
Oleg
> Optionally use JSSE inbuilt endpoint identification
> ---
>
> Key: HTTPCLIENT-2151
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2151
> Project: HttpComponents HttpClient
> Issue Type: Wish
> Components: HttpClient (classic)
>Reporter: Peter Dettman
>Priority: Minor
> Fix For: 5.4-alpha2
>
>
> This issue brings to the fore that part of HTTPCLIENT-2134 which is actually
> within the scope of HttpClient: the possibility of using JSSE inbuilt
> endpoint identification ("HTTPS") instead of performing those checks
> externally (typically in
> _org.apache.hc.client5.http.ssl.DefaultHostnameVerifier_).
> Java 1.7 added the method
> SSLParameters.setEndpointIdentificationAlgorithm(String). Calling this method
> with the argument "HTTPS" (case-insensitive) will result in the endpoint
> identification checks from RFC 2818 being automatically done by JSSE during
> the handshake (specifically when the X509TrustManager validates the server
> certificate). Note that the setting can also be interrogated via
> SSLParameters.getEndpointIdentificationAlgorithm.
> This has some minor advantages relative to performing the checks externally:
> * The check is done earlier in the handshake, reducing the cost of failed
> connections. (Technically this might still be achieved anyway by wrapping the
> X509TrustManager and applying a HostnameVerifier at that stage).
> * Per my comments in HTTPCLIENT-2134, in some nuanced situations SunJSSE will
> be willing to resume a session only if it knows for sure that endpoint
> identification is being used (which it can't know for an arbitrary
> HostnameVerifier).
> * SunJSSE's HTTPS endpoint identification has special (less-restrictive)
> rules for wildcard characters when the trust anchor is a certificate from the
> 'cacerts' file (what it calls a "public root CA").
> There may be other implications that I am unaware of, but I think this
> establishes that some users may prefer (or even need) to delegate endpoint
> identification to JSSE in this way. They presumably will not then want a
> duplicate check in DefaultHostnameVerifier.
> I am not yet that familiar with HttpClient, but I assume a user could already
> provide a custom SSLSocketFactory which enables JSSE endpoint identification
> on sockets before returning them, and then separately configure a no-op
> HostnameVerifier. I consider that to be rather fragile and it's easy to see
> mistakes leading to no endpoint identification being performed.
> I am proposing that HttpClient support this via a simpler configuration
> mechanism. To loosely outline some basic options:
> IGNORE: don't even check whether JSSE endpoint identification was enabled
> (i.e. assume it wasn't).
> AWARE: check at runtime whether JSSE endpoint identification was "HTTPS" for
> a given socket connection.
> DISABLE: remove any existing setting for JSSE endpoint identification (set it
> to "").
> FORCE: overwrite any existing setting for JSSE endpoint identification (set
> it to "HTTPS").
> So we would consider some value that came to us via the SSLSocketFactory's
> default socket setup, and whether to ignore it, respect it, or force it
> on/off. DefaultHostnameVerifier can then be set to possibly skip its own
> endpoint checks when appropriate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2134) HttpClient doesn't reuse TLS 1.2 Session
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810427#comment-17810427
]
Oleg Kalnichevski commented on HTTPCLIENT-2134:
---
[~peter.dettman] [~m_v_egorov] I was looking into HTTPCLIENT-2151 and concluded
that in order to be able to work on this new feature I needed a reasonable test
case. I went back to the original issue and was not able to reproduce it or
make any sense of what the problem actually was.
Please note I used just the core transport components simply because the core
has fewer levels of complexity on top of straight JSSE.
The ` sslParameters.setEndpointIdentificationAlgorithm("HTTPS")` line appears
to have no effect. I get pretty much identical TLS handshake (at least judging
by the TLS debug log output). I have not tried looking at the TLS packets with
WireShark yet. I need to know if the issue still valid before I do that.
I need a reasonable test case for HTTPCLIENT-2151 to proceed,
Oleg
{code:java}
final HttpRequester httpRequester = RequesterBootstrap.bootstrap()
.setSslSetupHandler(sslParameters -> {
sslParameters.setEndpointIdentificationAlgorithm("HTTPS"); // This
line does not seem to have any effect
sslParameters.setProtocols(new String[] {TLS.V_1_2.id});
})
.create();
final HttpHost target = new HttpHost("https", "sso.rbo.raiffeisen.ru");
final HttpCoreContext coreContext = HttpCoreContext.create();
final ClassicHttpRequest request = ClassicRequestBuilder.get()
.setPath("/")
.build();
try (ClassicHttpResponse response = httpRequester.execute(target, request,
Timeout.ofSeconds(5), coreContext)) {
System.out.println(request.getUri() + "->" + response.getCode());
System.out.println("==");
}
TimeUnit.SECONDS.sleep(1);
httpRequester.closeIdle(TimeValue.ZERO_MILLISECONDS);
try (ClassicHttpResponse response = httpRequester.execute(target, request,
Timeout.ofSeconds(5), coreContext)) {
System.out.println(request.getUri() + "->" + response.getCode());
System.out.println("==");
}
{code}
> HttpClient doesn't reuse TLS 1.2 Session
>
>
> Key: HTTPCLIENT-2134
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2134
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.13, 5.0.3
>Reporter: Maxim Egorov
>Priority: Minor
> Labels: volunteers-wanted
> Attachments: TestApacheHttpClientApp.java, handshake.log
>
>
> To reproduce run on java 11+:
> java -cp ... -Djavax.net.debug=ssl:handshake TestApacheHttpClientApp
> As you can see from handshake.log file HttpClient always create new tls
> session.
> The root of problem is support of Extended Master Key Extension in
> [https://github.com/openjdk/jdk/blob/jdk-11+28/src/java.base/share/classes/sun/security/ssl/ClientHello.java#L497.]
> The standard jdk HttpURLConnection doesn't be affected this issues because
> of it sets chc.sslConfig.identificationProtocol equals to HTTPS by default
> [https://github.com/openjdk/jdk/blob/jdk-11%2B28/src/java.base/share/classes/sun/net/www/protocol/https/HttpsClient.java#L560.]
> I tried to repeat the same trick (The commented code), but due to the bugs
> of JDK [https://bugs.openjdk.java.net/browse/JDK-8253368] and may be
> incorrect implementation of method
> org.apache.http.impl.BHttpConnectionBase.close it doesn't work.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPASYNC-171) Async http post with apache client 5.3
[
https://issues.apache.org/jira/browse/HTTPASYNC-171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807148#comment-17807148
]
Oleg Kalnichevski commented on HTTPASYNC-171:
-
[~tassadar] Of course, there is.
{code:java}
final PoolingAsyncClientConnectionManager cm =
PoolingAsyncClientConnectionManagerBuilder.create()
.setTlsConfigResolver(httphost -> TlsConfig.custom()
//.setVersionPolicy(HttpVersionPolicy.FORCE_HTTP_1)
//.setVersionPolicy(HttpVersionPolicy.FORCE_HTTP_2)
.setVersionPolicy(HttpVersionPolicy.NEGOTIATE)
.build())
.build();
try (final CloseableHttpAsyncClient client = HttpAsyncClients.custom()
.setConnectionManager(cm)
.build()) {
client.start();
final HttpHost target = new HttpHost("https", "httpbin.org");
final HttpClientContext clientContext = HttpClientContext.create();
final SimpleHttpRequest request = SimpleRequestBuilder.get()
.setHttpHost(target)
.setPath("/")
.build();
System.out.println("Executing request " + request);
final Future future = client.execute(
SimpleRequestProducer.create(request),
SimpleResponseConsumer.create(),
clientContext,
new FutureCallback() {
@Override
public void completed(final SimpleHttpResponse response) {
System.out.println(request + "->" + new StatusLine(response));
System.out.println("HTTP protocol " +
clientContext.getProtocolVersion());
final SSLSession sslSession = clientContext.getSSLSession();
if (sslSession != null) {
System.out.println("SSL protocol " +
sslSession.getProtocol());
System.out.println("SSL cipher suite " +
sslSession.getCipherSuite());
}
System.out.println(response.getBody());
}
@Override
public void failed(final Exception ex) {
System.out.println(request + "->" + ex);
}
@Override
public void cancelled() {
System.out.println(request + " cancelled");
}
});
future.get();
System.out.println("Shutting down");
client.close(CloseMode.GRACEFUL);
{code}
Oleg
> Async http post with apache client 5.3
> --
>
> Key: HTTPASYNC-171
> URL: https://issues.apache.org/jira/browse/HTTPASYNC-171
> Project: HttpComponents HttpAsyncClient
> Issue Type: Bug
>Reporter: tassadar
>Priority: Major
>
> I constructed an async client with:
> {code:java}
> private val httpclient = HttpAsyncClients.custom()
> .setConnectionManager(connectionManager)
> .build();{code}
> {color:#172b4d}And i noticed that the Content-Length header is not supplied.
> Invastigating i saw that all the requesta were made as http 2.0, so i tried
> to force an http 1.1 request:
> {color}
> {code:java}
> request.setVersion(HttpVersion.HTTP_1_1)
> localContext.setProtocolVersion(HttpVersion.HTTP_1_1){code}
> but it didn't worked, still the request were made as http 2.0. Trying to
> investigate further i noticed that even if the{color:#172b4d}
> H2RequestContent interceptor is able to manage the http 1.1 request
> (delegating it to super) the
> {color}AbstractH2StreamMultiplexer.{color:#172b4d}getProtocolVersion()
> involved in the process always returns {color}http_2:
> {code:java}
> @Override
> public ProtocolVersion getProtocolVersion() {
> return HttpVersion.HTTP_2;
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17807129#comment-17807129
]
Oleg Kalnichevski commented on HTTPCORE-763:
[~turing85] I cherry-picked the change-set to 5.2.x but I cannot promise a
release any time soon.
Oleg
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
> Fix For: 5.3-alpha2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET",
> URI.create("https://www.google.com//search/asdf?q=foobar;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCORE-763.
Resolution: Fixed
Committed to master.
Oleg
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
> Fix For: 5.3-alpha2
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET",
> URI.create("https://www.google.com//search/asdf?q=foobar;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski updated HTTPCORE-763:
---
Fix Version/s: 5.3-alpha2
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
> Fix For: 5.3-alpha2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET",
> URI.create("https://www.google.com//search/asdf?q=foobar;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806885#comment-17806885
]
Oleg Kalnichevski commented on HTTPCORE-763:
> Should we throw a {{new IllegalArgumentException("authority must be set");}}
> in this case?
[~turing85] No, we should not. The authority may be set by the protocol handler
depending on the execution pipeline. It may be null at the request construction
time but will be checked and enforced at the execution time. See
`RequestValidateHost` for reference. Theoretically the path check could also be
executed there and trigger a `ProtocolException` at execution time.
Oleg
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET",
> URI.create("https://www.google.com//search/asdf?q=foobar;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806870#comment-17806870
]
Oleg Kalnichevski edited comment on HTTPCORE-763 at 1/15/24 3:13 PM:
-
> If you give the go-ahead, then I can provide a PR.
[~turing85] A PR at GitHub is _the_ way to go.
Please also note that a valid HTTP request message (HTTP/1.1 and above) always
requires an authority. The request path must never start with a double slash
sequence.
Oleg
was (Author: olegk):
> If you give the go-ahead, then I can provide a PR.
[~turing85] A PR at GitHub is _the_ way to go.
Please also note that a valid HTTP request message always requires an
authority. The request path must never start with a double slash sequence.
Oleg
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET",
> URI.create("https://www.google.com//search/asdf?q=foobar;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806870#comment-17806870
]
Oleg Kalnichevski commented on HTTPCORE-763:
> If you give the go-ahead, then I can provide a PR.
[~turing85] A PR at GitHub is _the_ way to go.
Please also note that a valid HTTP request message always requires an
authority. The request path must never start with a double slash sequence.
Oleg
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET",
> URI.create("https://www.google.com//search/asdf?q=foobar;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-763) Behaviour of BasicHttpRequest wrt. path is inconsistent
[
https://issues.apache.org/jira/browse/HTTPCORE-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806859#comment-17806859
]
Oleg Kalnichevski commented on HTTPCORE-763:
[~turing85] Feel free to propose a fix.
Oleg
> Behaviour of BasicHttpRequest wrt. path is inconsistent
> ---
>
> Key: HTTPCORE-763
> URL: https://issues.apache.org/jira/browse/HTTPCORE-763
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.1, 5.2, 5.3-alpha1
>Reporter: Marco Bungart
>Priority: Minor
>
> The [Setter for {{path}} in
> {{BasicHttpRequest}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L210]
> checks whether the parameter {{path}} starts with two slashes and - if so -
> throws an {{IllegalArgumentException}} (through the call to
> [{{Args.check(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/util/Args.java#L39].
> Similarly, method
> [{{setUri(...)}}|https://github.com/apache/httpcomponents-core/blob/26b3bc6d6dc56587afdb12d84361009d55a5197f/httpcore5/src/main/java/org/apache/hc/core5/http/message/BasicHttpRequest.java#L264]
> checks the raw path from tur {{URI}}.
> This leads to inconsistent behaviour. A constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "//search/asdf?q=foobar");
> {code}
> works fine, while a constructor call like:
> {code:java}
> new BasicHttpRequest("GET", "http", URIAuthority.create("google.com"),
> "doesNotMatter")
> .setPath("//search/asdf?q=foobar");
> {code}
> or
> {code:java}
> new BasicHttpRequest("GET", URI.create("https://www.google.com//foo;));
> {code}
> throws an {{{}IllegalArgumentException{}}}.
> The Class {{BasicHttpRequest}} was extended in [commit
> {{1614d5d}}|https://github.com/apache/httpcomponents-core/commit/1614d5d5a5de6f4faadff9ecb566982d56b65179]
> and is present since.
> I think what should happen is that the path should only be checked for {{//}}
> if the {{authority}} is not set since [RFC 3986 in section
> 3|https://www.rfc-editor.org/rfc/rfc3986#section-3] specifies:
> {code}
>The scheme and path components are required, though the path may be
>empty (no characters). When authority is present, the path must
>either be empty or begin with a slash ("/") character. When
>authority is not present, the path cannot begin with two slash
>characters ("//").
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-1081) getMetrics() does not work for HEAD or other early release methods
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-1081. --- Fix Version/s: 5.0 Resolution: Fixed > getMetrics() does not work for HEAD or other early release methods > -- > > Key: HTTPCLIENT-1081 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1081 > Project: HttpComponents HttpClient > Issue Type: Bug >Reporter: Sebb >Priority: Major > Fix For: 5.4-alpha2, 5.0 > > > The following code works fine for GET: >HttpResponse httpResponse = > httpClient.execute(httpRequest, localContext); >HttpConnection conn = (HttpConnection) > localContext.getAttribute(ExecutionContext.HTTP_CONNECTION); >HttpConnectionMetrics metrics = conn.getMetrics(); > but fails on the getMetrics() call when the request is HEAD: >org.apache.http.impl.conn.ConnectionShutdownException >at > org.apache.http.impl.conn.AbstractClientConnAdapter.assertValid(AbstractClientConnAdapter.java:154) >at > org.apache.http.impl.conn.AbstractClientConnAdapter.getMetrics(AbstractClientConnAdapter.java:190) > This is because the getMetrics() method calls assertValid() which checks that > the connection has not been released, and the HEAD request is released as > soon as it completes, before returning to the caller. > Possible solutions: > - don't call assertValid() in getMetrics() > - store a pointer to the metrics in the local context -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1081) getMetrics() does not work for HEAD or other early release methods
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806449#comment-17806449
]
Oleg Kalnichevski commented on HTTPCLIENT-1081:
---
[~sebb] The issue has been resolved in 5.0 but I only now got around to
verifying that.
{code:java}
// Create local HTTP context
final HttpClientContext localContext = new HttpClientContext();
final HttpGet httpget = new HttpGet("http://httpbin.org/get;);
System.out.println("Executing request " + httpget.getMethod() + " " +
httpget.getUri());
// Pass local context as a parameter
httpclient.execute(httpget, localContext, response -> {
System.out.println("");
System.out.println(httpget + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
final EndpointDetails endpointDetails = localContext.getEndpointDetails();
System.out.println("Remote address: " + endpointDetails.getRemoteAddress());
System.out.println("Request counts: " + endpointDetails.getRequestCount());
System.out.println("Response counts: " +
endpointDetails.getResponseCount());
System.out.println("Bytes sent: " + endpointDetails.getSentBytesCount());
System.out.println("Bytes received: " +
endpointDetails.getReceivedBytesCount());
return null;
});
// EndpointDetails is present in the context even if the underlying connection
// gets immediately released back to the connection pool
final HttpHead httphead = new HttpHead("http://httpbin.org/get;);
System.out.println("Executing request " + httphead.getMethod() + " " +
httphead.getUri());
// Pass local context as a parameter
httpclient.execute(httphead, localContext, response -> {
System.out.println("");
System.out.println(httphead + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
final EndpointDetails endpointDetails = localContext.getEndpointDetails();
System.out.println("Remote address: " + endpointDetails.getRemoteAddress());
System.out.println("Request counts: " + endpointDetails.getRequestCount());
System.out.println("Response counts: " +
endpointDetails.getResponseCount());
System.out.println("Bytes sent: " + endpointDetails.getSentBytesCount());
System.out.println("Bytes received: " +
endpointDetails.getReceivedBytesCount());
return null;
});
{code}
Oleg
> getMetrics() does not work for HEAD or other early release methods
> --
>
> Key: HTTPCLIENT-1081
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1081
> Project: HttpComponents HttpClient
> Issue Type: Bug
>Reporter: Sebb
>Priority: Major
> Fix For: 5.4-alpha2
>
>
> The following code works fine for GET:
>HttpResponse httpResponse =
> httpClient.execute(httpRequest, localContext);
>HttpConnection conn = (HttpConnection)
> localContext.getAttribute(ExecutionContext.HTTP_CONNECTION);
>HttpConnectionMetrics metrics = conn.getMetrics();
> but fails on the getMetrics() call when the request is HEAD:
>org.apache.http.impl.conn.ConnectionShutdownException
>at
> org.apache.http.impl.conn.AbstractClientConnAdapter.assertValid(AbstractClientConnAdapter.java:154)
>at
> org.apache.http.impl.conn.AbstractClientConnAdapter.getMetrics(AbstractClientConnAdapter.java:190)
> This is because the getMetrics() method calls assertValid() which checks that
> the connection has not been released, and the HEAD request is released as
> soon as it completes, before returning to the caller.
> Possible solutions:
> - don't call assertValid() in getMetrics()
> - store a pointer to the metrics in the local context
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-1099) Overriding Caching Policies
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-1099. --- Fix Version/s: (was: 5.4-alpha2) Resolution: Won't Fix > Overriding Caching Policies > --- > > Key: HTTPCLIENT-1099 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1099 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.1.1 >Reporter: Bart Robeyns >Assignee: Jonathan Moore >Priority: Minor > Labels: cache, policy, stuck, volunteers-wanted > Attachments: OpenPolicies.patch > > > It is not possible to alter the behaviour of the CachingHttpClient because > the policies defining the behaviour are private and tied directly to specific > implementations in the CachingHttpClients constructor. Furthermore, these > policies are package private, discouraging reuse and/or extensions. > Making this possible is easy enough (provide some policy-setters or > -constructor-args in CachingHttpClient and make the policy-classes public); > the attached patch allows custom Policies, extending the default ones to be > set on the CacheConfig class. > The specific case that led to this question: > A back-end application only sets its Content-Length header for responses > below 8K. This response does get stored in the cache, but when retrieving it > from the cache, CacheValidityPolicy.contentLengthHeaderMatchesActualLength > checks the Content-Length header with the stored size (to verify whether the > cached content is complete). This check fails, causing the cache entry to be > deemed unusable. If we were able to provide our own subclassed > CacheValidityPolicy, it would be easy to skip the check if the header is > missing and thus accomodate this specific back-end quirk. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1099) Overriding Caching Policies
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17806281#comment-17806281
]
Oleg Kalnichevski commented on HTTPCLIENT-1099:
---
As of 5.4 one will be able to control the behavior of the caching layer by
using a custom execution context. At this point one can set request cache
control directives and interrogate the context to get details of the cache
operation and its outcome.
In the future we can consider adding pluggable custom strategy interfaces for
request / response cache suitability decisions but not until there is a strong
case for such API.
Oleg
{code:java}
final ClassicHttpRequest httpget1 = ClassicRequestBuilder.get()
.setHttpHost(target)
.setPath("/")
.build();
// Use default cache control
final HttpCacheContext context = CacheContextBuilder.create()
.setCacheControl(RequestCacheControl.DEFAULT)
.build();
System.out.println("Executing request " + httpget1.getMethod() + " " +
httpget1.getUri());
httpclient.execute(httpget1, context, response -> {
System.out.println("");
System.out.println(httpget1 + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
System.out.println("Cache status: " + context.getCacheResponseStatus());
System.out.println("Request cache control: " +
context.getRequestCacheControl());
System.out.println("Response cache control: " +
context.getResponseCacheControl());
final HttpCacheEntry cacheEntry = context.getCacheEntry();
if (cacheEntry != null) {
System.out.println("Cache entry resource: " + cacheEntry.getResource());
System.out.println("Date: " + cacheEntry.getInstant());
System.out.println("Expires: " + cacheEntry.getExpires());
System.out.println("Last modified: " + cacheEntry.getLastModified());
}
return null;
});
final ClassicHttpRequest httpget2 = ClassicRequestBuilder.get()
.setHttpHost(target)
.setPath("/")
.build();
// Ensure a custom freshness for the cache entry
context.setRequestCacheControl(RequestCacheControl.builder()
.setMinFresh(100)
.build());
System.out.println("Executing request " + httpget2.getMethod() + " " +
httpget2.getUri());
httpclient.execute(httpget2, context, response -> {
System.out.println("");
System.out.println(httpget2 + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
System.out.println("Cache status: " + context.getCacheResponseStatus());
System.out.println("Request cache control: " +
context.getRequestCacheControl());
System.out.println("Response cache control: " +
context.getResponseCacheControl());
final HttpCacheEntry cacheEntry = context.getCacheEntry();
if (cacheEntry != null) {
System.out.println("Cache entry resource: " + cacheEntry.getResource());
System.out.println("Date: " + cacheEntry.getInstant());
System.out.println("Expires: " + cacheEntry.getExpires());
System.out.println("Last modified: " + cacheEntry.getLastModified());
}
return null;
});
Thread.sleep(2000);
final ClassicHttpRequest httpget3 = ClassicRequestBuilder.get()
.setHttpHost(target)
.setPath("/")
.build();
// Try to force cache entry re-validation
context.setRequestCacheControl(RequestCacheControl.builder()
.setMaxAge(0)
.build());
System.out.println("Executing request " + httpget3.getMethod() + " " +
httpget3.getUri());
httpclient.execute(httpget3, context, response -> {
System.out.println("");
System.out.println(httpget3 + "->" + new StatusLine(response));
EntityUtils.consume(response.getEntity());
System.out.println("Cache status: " + context.getCacheResponseStatus());
System.out.println("Request cache control: " +
context.getRequestCacheControl());
System.out.println("Response cache control: " +
context.getResponseCacheControl());
final HttpCacheEntry cacheEntry = context.getCacheEntry();
if (cacheEntry != null) {
System.out.println("Cache entry resource: " + cacheEntry.getResource());
System.out.println("Date: " + cacheEntry.getInstant());
System.out.println("Expires: " + cacheEntry.getExpires());
System.out.println("Last modified: " + cacheEntry.getLastModified());
}
return null;
});
{code}
> Overriding Caching Policies
> ---
>
> Key: HTTPCLIENT-1099
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1099
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpCache
>Affects Versions: 4.1.1
>Reporter: Bart Robeyns
>Assignee: Jonathan Moore
>
[jira] [Resolved] (HTTPCLIENT-2315) Inconsistent observance of system properties when using HttpClients.createSystem
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Kalnichevski resolved HTTPCLIENT-2315. --- Fix Version/s: 5.4-alpha2 Resolution: Fixed > Inconsistent observance of system properties when using > HttpClients.createSystem > > > Key: HTTPCLIENT-2315 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2315 > Project: HttpComponents HttpClient > Issue Type: Bug > Components: HttpClient (classic) >Affects Versions: 5.3 >Reporter: Vitor Dantas >Priority: Minor > Fix For: 5.3.1, 5.4-alpha2 > > > During the startup of my application, I configure an SSLContext with > additional certificates and set it as a global default using > SSLContext.setDefault. Subsequently, all requests created without explicit > configuration can successfully connect to servers requiring these > certificates. This setup functions seamlessly with various APIs I currently > utilize: > * java.net.URLConnection (e.g., URLConnection conn = url.openConnection()) > * java.net.http.HttpClient (e.g., HttpClient client = > HttpClient.newHttpClient()) > * org.springframework.ws.client.core.WebServiceTemplate (e.g., > WebServiceTemplate webServiceTemplate = new WebServiceTemplate()) > Previously, this approach worked flawlessly with > org.apache.httpcomponents:httpclient:4.5.14, where clients were created using: > HttpClient httpclient = > org.apache.http.impl.client.HttpClients.createSystem(); > However, issues arise (due to missing certificates) after upgrading to > org.apache.httpcomponents.client5:httpclient5:5.3 and creating clients with: > HttpClient httpclient = > org.apache.hc.client5.http.impl.classic.HttpClients.createSystem(); > When creating a client with > HttpClientBuilder.create().useSystemProperties().build() or more succinctly > with HttpClients.createSystem(), the expectation is that system properties > are observed throughout. > However, upon inspecting the code, it becomes evident that the connection > manager underneath is created using > PoolingHttpClientConnectionManagerBuilder.create().build(), and not > PoolingHttpClientConnectionManagerBuilder.create().useSystemProperties().build() > as one might anticipate. > Consequently, an internal condition registers the HTTPS socketFactory using > SSLConnectionSocketFactory.getSocketFactory() instead of > SSLConnectionSocketFactory.getSystemSocketFactory(). This, in turn, results > in the creation of a new SSLContext.getInstance(SSLContextBuilder.TLS) > instead of leveraging the default one through SSLSocketFactory.getDefault(). > Why is this the case? Is it possible to change this behavior so that the > configuration is simplified when one aims to use the default socket factory? > Specifically, without the need for explicit configuration of the connection > manager in such a common scenario. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2315) Inconsistent observance of system properties when using HttpClients.createSystem
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17804274#comment-17804274 ] Oleg Kalnichevski commented on HTTPCLIENT-2315: --- [~vitorcd] Please review / test the proposed fix: [https://github.com/apache/httpcomponents-client/compare/master..HTTPCLIENT-2315] Oleg > Inconsistent observance of system properties when using > HttpClients.createSystem > > > Key: HTTPCLIENT-2315 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2315 > Project: HttpComponents HttpClient > Issue Type: Bug > Components: HttpClient (classic) >Affects Versions: 5.3 >Reporter: Vitor Dantas >Priority: Minor > Fix For: 5.3.1 > > > During the startup of my application, I configure an SSLContext with > additional certificates and set it as a global default using > SSLContext.setDefault. Subsequently, all requests created without explicit > configuration can successfully connect to servers requiring these > certificates. This setup functions seamlessly with various APIs I currently > utilize: > * java.net.URLConnection (e.g., URLConnection conn = url.openConnection()) > * java.net.http.HttpClient (e.g., HttpClient client = > HttpClient.newHttpClient()) > * org.springframework.ws.client.core.WebServiceTemplate (e.g., > WebServiceTemplate webServiceTemplate = new WebServiceTemplate()) > Previously, this approach worked flawlessly with > org.apache.httpcomponents:httpclient:4.5.14, where clients were created using: > HttpClient httpclient = > org.apache.http.impl.client.HttpClients.createSystem(); > However, issues arise (due to missing certificates) after upgrading to > org.apache.httpcomponents.client5:httpclient5:5.3 and creating clients with: > HttpClient httpclient = > org.apache.hc.client5.http.impl.classic.HttpClients.createSystem(); > When creating a client with > HttpClientBuilder.create().useSystemProperties().build() or more succinctly > with HttpClients.createSystem(), the expectation is that system properties > are observed throughout. > However, upon inspecting the code, it becomes evident that the connection > manager underneath is created using > PoolingHttpClientConnectionManagerBuilder.create().build(), and not > PoolingHttpClientConnectionManagerBuilder.create().useSystemProperties().build() > as one might anticipate. > Consequently, an internal condition registers the HTTPS socketFactory using > SSLConnectionSocketFactory.getSocketFactory() instead of > SSLConnectionSocketFactory.getSystemSocketFactory(). This, in turn, results > in the creation of a new SSLContext.getInstance(SSLContextBuilder.TLS) > instead of leveraging the default one through SSLSocketFactory.getDefault(). > Why is this the case? Is it possible to change this behavior so that the > configuration is simplified when one aims to use the default socket factory? > Specifically, without the need for explicit configuration of the connection > manager in such a common scenario. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2314) IllegalStateException: Endpoint is not connected
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCLIENT-2314.
---
Fix Version/s: 5.4-alpha2
Resolution: Fixed
Fix committed to 5.3.x and master.
Oleg
> IllegalStateException: Endpoint is not connected
>
>
> Key: HTTPCLIENT-2314
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2314
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.2, 5.3
>Reporter: Philip Helger
>Priority: Minor
> Fix For: 5.3.1, 5.4-alpha2
>
> Attachments: debuglog-httpclient-2314.txt, httpclient-2314-1.txt,
> httpclient-2314-2.txt
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Using `PoolingHttpClientConnectionManager`, under high traffic situations it
> occurs consistently that the exception "Endpoint is not connected" is invoked.
>
> The full stacktrace looks like this:
> {{Caused by: java.lang.IllegalStateException: Endpoint is not connected}}
> {{ at org.apache.hc.core5.util.Asserts.check(Asserts.java:38)
> ~[httpcore5-5.2.4.jar:5.2.4]}}
> {{ at
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.getValidatedPoolEntry(PoolingHttpClientConnectionManager.java:664)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.setSocketTimeout(PoolingHttpClientConnectionManager.java:697)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:211)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:116)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:245)
> ~[httpclient5-5.3.jar:5.3]}}
> {{ at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:188)
> ~[httpclient5-5.3.jar:5.3]}}
>
> Unfortunately I don't have a standalone example to reproduce it.
> Any suggestions for a workaround
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
