[jira] [Commented] (HTTPCLIENT-2323) When using HttpClientBuilder without setting a user agent an expensive operation seems to be used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17821978#comment-17821978
]
Arturo Bernal commented on HTTPCLIENT-2323:
---
Hi [~winstonwaite]
It is unlikely that the 4.x line will be updated beyond critical security
updates.
Please consider updating to 5.x and proceed from ther
Thank you.
> When using HttpClientBuilder without setting a user agent an expensive
> operation seems to be used
> -
>
> Key: HTTPCLIENT-2323
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2323
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.12
> Environment: Docker image using maven:3.9.1-eclipse-temurin-17-focal
> running in kubernetes
>Reporter: Rob
>Priority: Minor
> Attachments: Screenshot 2024-02-28 at 12.51.41 PM.png
>
>
> We have an application that has a fairly high outbound http call rate using
> Apache Http Client. We have been profiling it recently using the async
> profiler and I noticed that almost 10% of our cpu time is spent in
> VersionInfo.getUserAgent.
> We use HttpClientBuilder for each call (this seems the correct way to be able
> to use different connection pools and settings).
> I am guessing because we do not explicitly set the user agent that the client
> will go determine the client version and java version and use this... the
> automatically generated user agent in our case looks like:
> {code:java}
> User-Agent: Apache-HttpClient/4.5.12 (Java/17.0.7){code}
> I have attached the profiler flame graph. I would imagine something like this
> could be checked once and used for any further calls. I have not tested it
> yet but I am hoping a workaround would be to make sure to set a user agent
> and then none of this classloader stuff would need to happen for each call.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2282) Handle java.net.SocketException: Connection reset more gracefully
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2282: -- Description: Currently in edge cases the retry mechanism leveraging DefaultHttpRequestRetryHandler does not catch "Connection reset" exceptions. This is caused by # A connection pool which keeps the connections alive for too long # The reduced default validation of connections in [https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html:] "The code now only checks the connection if the elapsed time since the last use of the connection exceeds the timeout that has been set. The default timeout is set to 2000ms" The error scenario is as follows: # A client opens a number of connection almost in parallel which exceed the number of retries # then the connections are reused shortly before they are closed silently by some proxy or the server. # Now the client tries to reuse those but runs into the "Connection reset" issue. Although it will retry through DefaultHttpRequestRetryHandler 3 different pooled connections they all suffer from the fact that they are not longer accepted (and therefore throw the same SocketException). Maybe one could improve the resilience by calling [https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html#setValidateAfterInactivity(int]) with 0 on the underlying connection pool manager until a connection is successfully established to that host again. was: Currently in edge cases the retry mechanism leveraging DefaultHttpRequestRetryHandler does not catch "Connection reset" exceptions. This is caused by # A connection pool which keeps the connections alive for too long # The reduced default validation of connections in https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html: "The code now only checks the connection if the elapsed time since the last use of the connection exceeds the timeout that has been set. The default timeout is set to 2000ms" The error scenario is as follows: # A client opens a number of connection almost in parallel which exceed the number of retries # then the connections are reused shortly before they are closed silently by some proxy or the server. # Now the client tries to reuse those but runs into the "Connection reset" issue. Although it will retry through DefaultHttpRequestRetryHandler 3 different pooled connections they all suffer from the fact that they are not longer accepted (and therefore throw the same SocketException). Maybe one could improve the resilience by calling https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html#setValidateAfterInactivity(int) with 0 on the underlying connection pool manager until a connection is successfully established to that host again. > Handle java.net.SocketException: Connection reset more gracefully > - > > Key: HTTPCLIENT-2282 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2282 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Affects Versions: 4.5.14 >Reporter: Konrad Windszus >Priority: Major > Labels: volunteers-wanted > Fix For: Stuck > > > Currently in edge cases the retry mechanism leveraging > DefaultHttpRequestRetryHandler does not catch "Connection reset" exceptions. > This is caused by > # A connection pool which keeps the connections alive for too long > # The reduced default validation of connections in > [https://hc.apache.org/httpcomponents-client-4.5.x/current/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html:] > "The code now only checks the connection if the elapsed time since the last > use of the connection exceeds the timeout that has been set. The default > timeout is set to 2000ms" > The error scenario is as follows: > # A client opens a number of connection almost in parallel which exceed the > number of retries > # then the connections are reused shortly before they are closed silently by > some proxy or the server. > # Now the client tries to reuse those but runs into the "Connection reset" > issue. Although it will retry through DefaultHttpRequestRetryHandler 3 > different pooled connections they all suffer from the fact that they are not > longer accepted (and therefore throw the same SocketException). > Maybe one could improve the resilience by calling >
[jira] [Assigned] (HTTPCLIENT-2319) DateUtils.parseStandardDate() throws exception instead of returning null
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal reassigned HTTPCLIENT-2319:
-
Assignee: Arturo Bernal
> DateUtils.parseStandardDate() throws exception instead of returning null
>
>
> Key: HTTPCLIENT-2319
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2319
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Bernd Watermann
>Assignee: Arturo Bernal
>Priority: Minor
> Fix For: 5.4-alpha2
>
>
> The following line of code
> {code:java}
> org.apache.hc.client5.http.utils.DateUtils.parseStandardDate("Thu Feb 22
> 17:20:18 2024");{code}
> throws an exception with the following stack trace:
> {noformat}
> java.time.DateTimeException: Unable to obtain Instant from TemporalAccessor:
> {},ISO resolved to 2024-02-22T17:20:18 of type java.time.format.Parsed
> at java.base/java.time.Instant.from(Instant.java:380)
> at
> org.apache.hc.client5.http.utils.DateUtils.parseDate(DateUtils.java:165)
> at
> org.apache.hc.client5.http.utils.DateUtils.parseStandardDate(DateUtils.java:183){noformat}
> The documentation for parseStandardDate(), however, says:
> ??Returns: the parsed instant or null if input could not be parsed??
> Throwing an exception does not seem to conform to that spec.
> Note that I don't care (at present) if the given string should actually be
> parseable or not. The primary issue is error handling and the fact that any
> error should lead to a null result and not to an exception being thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Assigned] (HTTPCLIENT-2318) Implement HttpClientConnectionManager#isShutdown
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal reassigned HTTPCLIENT-2318: - Assignee: Arturo Bernal > Implement HttpClientConnectionManager#isShutdown > > > Key: HTTPCLIENT-2318 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2318 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Reporter: Edmund Ham >Assignee: Arturo Bernal >Priority: Major > Fix For: 5.4-alpha2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hi all, > We have an internal library that has core logics and lots of microservices > are using it. Within the internal library, we have "HttpClientPool" where we > are caching different HttpClient based on keys and microservices are using > them. Recently, we've been seeing some of OOM errors on microservices when > using HttpClients from our "HttpClientPool". > From https://issues.apache.org/jira/browse/HTTPCLIENT-2039 and > https://issues.apache.org/jira/browse/HTTPCLIENT-1924 we understand that it > is the intention to shutdown the connection pool in case of Java Errors. I'm > now wondering if it's possible to have new API, > "HttpClientConnectionManager#isShutdown" so that we as a library can identify > if the connection pool is closed or not, before giving out to our clients? > Currently there seems to be no way to identify it, unless we actually try to > make an outbound call and get "IllegalStateException: Connection pool shut > down" exception message, which makes impossible for us to do because we as a > library don't want to make additional HTTP client call every time when our > client requests for the HttpClient instance. > Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Assigned] (HTTPCLIENT-2318) Implement HttpClientConnectionManager#isShutdown
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal reassigned HTTPCLIENT-2318: - Assignee: (was: Arturo Bernal) > Implement HttpClientConnectionManager#isShutdown > > > Key: HTTPCLIENT-2318 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2318 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Reporter: Edmund Ham >Priority: Major > Fix For: 5.4-alpha2 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Hi all, > We have an internal library that has core logics and lots of microservices > are using it. Within the internal library, we have "HttpClientPool" where we > are caching different HttpClient based on keys and microservices are using > them. Recently, we've been seeing some of OOM errors on microservices when > using HttpClients from our "HttpClientPool". > From https://issues.apache.org/jira/browse/HTTPCLIENT-2039 and > https://issues.apache.org/jira/browse/HTTPCLIENT-1924 we understand that it > is the intention to shutdown the connection pool in case of Java Errors. I'm > now wondering if it's possible to have new API, > "HttpClientConnectionManager#isShutdown" so that we as a library can identify > if the connection pool is closed or not, before giving out to our clients? > Currently there seems to be no way to identify it, unless we actually try to > make an outbound call and get "IllegalStateException: Connection pool shut > down" exception message, which makes impossible for us to do because we as a > library don't want to make additional HTTP client call every time when our > client requests for the HttpClient instance. > Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2319) DateUtils.parseStandardDate() throws exception instead of returning null
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal resolved HTTPCLIENT-2319.
---
Fix Version/s: 5.4-alpha2
Resolution: Fixed
> DateUtils.parseStandardDate() throws exception instead of returning null
>
>
> Key: HTTPCLIENT-2319
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2319
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.3.1
>Reporter: Bernd Watermann
>Priority: Minor
> Fix For: 5.4-alpha2
>
>
> The following line of code
> {code:java}
> org.apache.hc.client5.http.utils.DateUtils.parseStandardDate("Thu Feb 22
> 17:20:18 2024");{code}
> throws an exception with the following stack trace:
> {noformat}
> java.time.DateTimeException: Unable to obtain Instant from TemporalAccessor:
> {},ISO resolved to 2024-02-22T17:20:18 of type java.time.format.Parsed
> at java.base/java.time.Instant.from(Instant.java:380)
> at
> org.apache.hc.client5.http.utils.DateUtils.parseDate(DateUtils.java:165)
> at
> org.apache.hc.client5.http.utils.DateUtils.parseStandardDate(DateUtils.java:183){noformat}
> The documentation for parseStandardDate(), however, says:
> ??Returns: the parsed instant or null if input could not be parsed??
> Throwing an exception does not seem to conform to that spec.
> Note that I don't care (at present) if the given string should actually be
> parseable or not. The primary issue is error handling and the fact that any
> error should lead to a null result and not to an exception being thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Assigned] (HTTPCLIENT-2318) Implement HttpClientConnectionManager#isShutdown
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal reassigned HTTPCLIENT-2318: - Assignee: Arturo Bernal > Implement HttpClientConnectionManager#isShutdown > > > Key: HTTPCLIENT-2318 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2318 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Reporter: Edmund Ham >Assignee: Arturo Bernal >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Hi all, > We have an internal library that has core logics and lots of microservices > are using it. Within the internal library, we have "HttpClientPool" where we > are caching different HttpClient based on keys and microservices are using > them. Recently, we've been seeing some of OOM errors on microservices when > using HttpClients from our "HttpClientPool". > From https://issues.apache.org/jira/browse/HTTPCLIENT-2039 and > https://issues.apache.org/jira/browse/HTTPCLIENT-1924 we understand that it > is the intention to shutdown the connection pool in case of Java Errors. I'm > now wondering if it's possible to have new API, > "HttpClientConnectionManager#isShutdown" so that we as a library can identify > if the connection pool is closed or not, before giving out to our clients? > Currently there seems to be no way to identify it, unless we actually try to > make an outbound call and get "IllegalStateException: Connection pool shut > down" exception message, which makes impossible for us to do because we as a > library don't want to make additional HTTP client call every time when our > client requests for the HttpClient instance. > Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1378) CacheableRequestPolicy and ResponseCachingPolicy Have Overlapping Functionality
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790776#comment-17790776 ] Arturo Bernal commented on HTTPCLIENT-1378: --- I suppose this ticket can be closed after the release of version 5.4.x. > CacheableRequestPolicy and ResponseCachingPolicy Have Overlapping > Functionality > --- > > Key: HTTPCLIENT-1378 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1378 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.3 Beta2 >Reporter: James Leigh >Priority: Minor > Labels: stuck, volunteers-wanted > Fix For: Stuck > > > As discussed in HTTPCLIENT-1370, both classes check the same thing in the > request object for cache-ability, but handle edge cases differently. The > repeated code should be removed and either CacheableRequestPolicy injected as > a dependency into the ResponseCachingPolicy or the two classes should be > merged into a single class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2309) Proxy url set via HttpClientBuilder is not working
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789451#comment-17789451
]
Arturo Bernal commented on HTTPCLIENT-2309:
---
Isn't this issue the same as the one reported in HTTPCLIENT-2307 ?
> Proxy url set via HttpClientBuilder is not working
> --
>
> Key: HTTPCLIENT-2309
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2309
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.2.2
>Reporter: Marcin Dąbrowski
>Priority: Major
>
> I'm using {{org.apache.hc.client5.http.impl.classic.HttpClientBuilder}} and
> setting proxy via {{builder.setProxy(HttpHost.create(myProxyUri))}}.
> After upgrade from 5.2.1 to 5.2.2 I'm getting:
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Invalid Proxy
> at java.base/java.net.Socket.(Socket.java:177)
> at
> org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory.createSocket(SSLConnectionSocketFactory.java:208)
> at
> org.apache.hc.client5.http.impl.io.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:158)
> at
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:447)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:162)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.connectEndpoint(InternalExecRuntime.java:172)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:146)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:96)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:152)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:115)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2301) BasicHttpClientConnectionManager returns response of previous requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal updated HTTPCLIENT-2301:
--
Fix Version/s: 5.4-alpha1
(was: Stuck)
> BasicHttpClientConnectionManager returns response of previous requests
> --
>
> Key: HTTPCLIENT-2301
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2301
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.2.1, 5.3-alpha1
> Environment: JAVA 18
> kotlin 1.7.22
> Ubuntu 22.04 x64
>Reporter: Pierre N.
>Assignee: Arturo Bernal
>Priority: Minor
> Labels: volunteers-wanted
> Fix For: 5.4-alpha1
>
> Attachments: test.kt
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> BasicHttpClientConnectionManager doesn't handle request timeout correctly and
> return the response of a previous request.
> The unit test below should output:
> {quote}
> http://localhost:41751/1 -> 1
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> {quote}
> But instead it returns:
> {quote}
> http://localhost:33875/1 -> 1
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> 2
> http://localhost:33875/3 -> 3
> {quote}
> As you can see it returns 2 when requesting the uri /3 which returned 3.
> Also it timeout on the first request to /3 while it shouldn't.
> Replace with PoolingHttpClientConnectionManager() and it works as expected.
> Kotlin unit test to reproduce :
> {quote}
> import com.github.tomakehurst.wiremock.client.WireMock.*
> import com.github.tomakehurst.wiremock.junit5.WireMockRuntimeInfo
> import com.github.tomakehurst.wiremock.junit5.WireMockTest
> import org.apache.hc.client5.http.classic.methods.HttpGet
> import org.apache.hc.client5.http.config.RequestConfig
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient
> import org.apache.hc.client5.http.impl.classic.HttpClients
> import org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager
> import org.apache.hc.core5.http.io.entity.EntityUtils
> import org.apache.hc.core5.util.Timeout
> import org.junit.jupiter.api.Test
> @WireMockTest
> internal class HttpClientRaceConditionDebug {
> @Test
> fun `debugging z`(wm: WireMockRuntimeInfo) {
> stubFor(get(urlEqualTo("/1")).willReturn(aResponse().withBody("1")))
>
> stubFor(get(urlEqualTo("/2")).willReturn(aResponse().withFixedDelay(2000).withBody("2")))
> stubFor(get(urlEqualTo("/3")).willReturn(aResponse().withBody("3")))
> val client = HttpClients.custom()
> .setConnectionManager(BasicHttpClientConnectionManager())
> //.setConnectionManager(PoolingHttpClientConnectionManager())
> .setDefaultRequestConfig(
> RequestConfig.custom()
> .setResponseTimeout(Timeout.ofSeconds(1))
> .build()
> )
> .build()
> client.executeAndLog("${wm.httpBaseUrl}/1")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> }
> private fun CloseableHttpClient.executeAndLog(uri: String) {
> try {
> execute(HttpGet(uri)) { println("$uri ->
> ${EntityUtils.toString(it.entity)}") }
> } catch (ex: Exception) {
> println("$uri -> $ex")
> }
> }
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2301) BasicHttpClientConnectionManager returns response of previous requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal resolved HTTPCLIENT-2301.
---
Resolution: Fixed
Fixed in 5.4 branch
> BasicHttpClientConnectionManager returns response of previous requests
> --
>
> Key: HTTPCLIENT-2301
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2301
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.2.1, 5.3-alpha1
> Environment: JAVA 18
> kotlin 1.7.22
> Ubuntu 22.04 x64
>Reporter: Pierre N.
>Assignee: Arturo Bernal
>Priority: Minor
> Labels: volunteers-wanted
> Fix For: 5.4-alpha1
>
> Attachments: test.kt
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> BasicHttpClientConnectionManager doesn't handle request timeout correctly and
> return the response of a previous request.
> The unit test below should output:
> {quote}
> http://localhost:41751/1 -> 1
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> {quote}
> But instead it returns:
> {quote}
> http://localhost:33875/1 -> 1
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> 2
> http://localhost:33875/3 -> 3
> {quote}
> As you can see it returns 2 when requesting the uri /3 which returned 3.
> Also it timeout on the first request to /3 while it shouldn't.
> Replace with PoolingHttpClientConnectionManager() and it works as expected.
> Kotlin unit test to reproduce :
> {quote}
> import com.github.tomakehurst.wiremock.client.WireMock.*
> import com.github.tomakehurst.wiremock.junit5.WireMockRuntimeInfo
> import com.github.tomakehurst.wiremock.junit5.WireMockTest
> import org.apache.hc.client5.http.classic.methods.HttpGet
> import org.apache.hc.client5.http.config.RequestConfig
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient
> import org.apache.hc.client5.http.impl.classic.HttpClients
> import org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager
> import org.apache.hc.core5.http.io.entity.EntityUtils
> import org.apache.hc.core5.util.Timeout
> import org.junit.jupiter.api.Test
> @WireMockTest
> internal class HttpClientRaceConditionDebug {
> @Test
> fun `debugging z`(wm: WireMockRuntimeInfo) {
> stubFor(get(urlEqualTo("/1")).willReturn(aResponse().withBody("1")))
>
> stubFor(get(urlEqualTo("/2")).willReturn(aResponse().withFixedDelay(2000).withBody("2")))
> stubFor(get(urlEqualTo("/3")).willReturn(aResponse().withBody("3")))
> val client = HttpClients.custom()
> .setConnectionManager(BasicHttpClientConnectionManager())
> //.setConnectionManager(PoolingHttpClientConnectionManager())
> .setDefaultRequestConfig(
> RequestConfig.custom()
> .setResponseTimeout(Timeout.ofSeconds(1))
> .build()
> )
> .build()
> client.executeAndLog("${wm.httpBaseUrl}/1")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> }
> private fun CloseableHttpClient.executeAndLog(uri: String) {
> try {
> execute(HttpGet(uri)) { println("$uri ->
> ${EntityUtils.toString(it.entity)}") }
> } catch (ex: Exception) {
> println("$uri -> $ex")
> }
> }
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Assigned] (HTTPCLIENT-2301) BasicHttpClientConnectionManager returns response of previous requests
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal reassigned HTTPCLIENT-2301:
-
Assignee: Arturo Bernal
> BasicHttpClientConnectionManager returns response of previous requests
> --
>
> Key: HTTPCLIENT-2301
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2301
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 5.2.1, 5.3-alpha1
> Environment: JAVA 18
> kotlin 1.7.22
> Ubuntu 22.04 x64
>Reporter: Pierre N.
>Assignee: Arturo Bernal
>Priority: Minor
> Labels: volunteers-wanted
> Fix For: Stuck
>
> Attachments: test.kt
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> BasicHttpClientConnectionManager doesn't handle request timeout correctly and
> return the response of a previous request.
> The unit test below should output:
> {quote}
> http://localhost:41751/1 -> 1
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> http://localhost:41751/3 -> 3
> {quote}
> But instead it returns:
> {quote}
> http://localhost:33875/1 -> 1
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/2 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> java.net.SocketTimeoutException: Read timed out
> http://localhost:33875/3 -> 2
> http://localhost:33875/3 -> 3
> {quote}
> As you can see it returns 2 when requesting the uri /3 which returned 3.
> Also it timeout on the first request to /3 while it shouldn't.
> Replace with PoolingHttpClientConnectionManager() and it works as expected.
> Kotlin unit test to reproduce :
> {quote}
> import com.github.tomakehurst.wiremock.client.WireMock.*
> import com.github.tomakehurst.wiremock.junit5.WireMockRuntimeInfo
> import com.github.tomakehurst.wiremock.junit5.WireMockTest
> import org.apache.hc.client5.http.classic.methods.HttpGet
> import org.apache.hc.client5.http.config.RequestConfig
> import org.apache.hc.client5.http.impl.classic.CloseableHttpClient
> import org.apache.hc.client5.http.impl.classic.HttpClients
> import org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager
> import org.apache.hc.core5.http.io.entity.EntityUtils
> import org.apache.hc.core5.util.Timeout
> import org.junit.jupiter.api.Test
> @WireMockTest
> internal class HttpClientRaceConditionDebug {
> @Test
> fun `debugging z`(wm: WireMockRuntimeInfo) {
> stubFor(get(urlEqualTo("/1")).willReturn(aResponse().withBody("1")))
>
> stubFor(get(urlEqualTo("/2")).willReturn(aResponse().withFixedDelay(2000).withBody("2")))
> stubFor(get(urlEqualTo("/3")).willReturn(aResponse().withBody("3")))
> val client = HttpClients.custom()
> .setConnectionManager(BasicHttpClientConnectionManager())
> //.setConnectionManager(PoolingHttpClientConnectionManager())
> .setDefaultRequestConfig(
> RequestConfig.custom()
> .setResponseTimeout(Timeout.ofSeconds(1))
> .build()
> )
> .build()
> client.executeAndLog("${wm.httpBaseUrl}/1")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/2")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> client.executeAndLog("${wm.httpBaseUrl}/3")
> }
> private fun CloseableHttpClient.executeAndLog(uri: String) {
> try {
> execute(HttpGet(uri)) { println("$uri ->
> ${EntityUtils.toString(it.entity)}") }
> } catch (ex: Exception) {
> println("$uri -> $ex")
> }
> }
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2290) Refactoring of synchronized sections to use lock primitives in light of virtual threads
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal resolved HTTPCLIENT-2290.
---
Resolution: Fixed
> Refactoring of synchronized sections to use lock primitives in light of
> virtual threads
> ---
>
> Key: HTTPCLIENT-2290
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2290
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Arturo Bernal
>Priority: Minor
> Fix For: 5.4-alpha1
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With the upcoming inclusion of virtual threads in JDK 21, we need to review
> and refactor sections of the HttpClient codebase that use the
> {{synchronized}} keyword.
> As virtual threads might introduce performance and deadlock issues with the
> current {{synchronized}} sections, we'll move towards using explicit lock
> primitives provided by the {{java.util.concurrent.locks}} package, such as
> {{{}ReentrantLock{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2290) Refactoring of synchronized sections to use lock primitives in light of virtual threads
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal updated HTTPCLIENT-2290:
--
Fix Version/s: 5.4-alpha1
> Refactoring of synchronized sections to use lock primitives in light of
> virtual threads
> ---
>
> Key: HTTPCLIENT-2290
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2290
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Arturo Bernal
>Priority: Minor
> Fix For: 5.4-alpha1
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With the upcoming inclusion of virtual threads in JDK 21, we need to review
> and refactor sections of the HttpClient codebase that use the
> {{synchronized}} keyword.
> As virtual threads might introduce performance and deadlock issues with the
> current {{synchronized}} sections, we'll move towards using explicit lock
> primitives provided by the {{java.util.concurrent.locks}} package, such as
> {{{}ReentrantLock{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776296#comment-17776296 ] Arturo Bernal commented on HTTPCORE-756: HI [~olegk] I'm glad to hear that.Fine by me to close the ticket. Ty > HttpCore HTTP Semantics conformance to RFC 9110 > --- > > Key: HTTPCORE-756 > URL: https://issues.apache.org/jira/browse/HTTPCORE-756 > Project: HttpComponents HttpCore > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > Time Spent: 7h 20m > Remaining Estimate: 0h > > Ensure HttpCore conforms to RFC 9110. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2293) HttpClient HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776294#comment-17776294 ] Arturo Bernal commented on HTTPCLIENT-2293: --- HI [~olegk] I'm glad to hear that.Fine by me to close the ticket. Ty > HttpClient HTTP Semantics conformance to RFC 9110 > - > > Key: HTTPCLIENT-2293 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2293 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2303) a shaded version of apache http client
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774663#comment-17774663 ] Arturo Bernal commented on HTTPCLIENT-2303: --- Shading can complicate matters and should be used as a last resort. It can cause more problems than it solves, especially for ubiquitous libraries like logging frameworks where multiple components might rely on them. It's usually better to manage dependencies properly to avoid conflicts. > a shaded version of apache http client > --- > > Key: HTTPCLIENT-2303 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2303 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Reporter: Mark Zitnik >Priority: Minor > > Hi Apache HTTP Client team, > I am one of the developers of > [clickhouse-java|https://github.com/ClickHouse/clickhouse-java/tree/v0.5.0]. > Recently, we decided to use Apache HTTP Client as our default HTTP driver. We > have a shaded version of our driver to avoid jar conflicts when running our > library. Any plans to add a shaded version of Apache HTTP Client? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCORE-759) Content-Length missing on null entity request content
[ https://issues.apache.org/jira/browse/HTTPCORE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770393#comment-17770393 ] Arturo Bernal edited comment on HTTPCORE-759 at 9/29/23 11:36 AM: -- I agree with your [~michael-o] , in the interpretation of [RFC 9110|https://www.rfc-editor.org/rfc/rfc9110#name-content-length:]. It clearly states that a 'Content-Length' header should not be sent when the request doesn't contain content and the method doesn't anticipate such data. was (Author: abernal): I agree with your [~michael-o] , in the interpretation of RFC 9110. It clearly states that a 'Content-Length' header should not be sent when the request doesn't contain content and the method doesn't anticipate such data. > Content-Length missing on null entity request content > - > > Key: HTTPCORE-759 > URL: https://issues.apache.org/jira/browse/HTTPCORE-759 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.2 >Reporter: Billy Jaime Beltran >Priority: Minor > Labels: easyfix > Time Spent: 10m > Remaining Estimate: 0h > > When making a POST request with an empty body, the following difference in > behaviour was observed. > In HttpComponents 4.4.x, a null body causes a Content-Length zero header to > be added. > In HttpComponents 5.2.x, a null body causes no Content-Length headers being > added. > While upgrading an Apache Camel application, we noticed that this breaks > calls to an upstream IIS server which replies with 411 (Length Required). > That server expects either a Content-Length 0 or a Transfer-Encoding: chunked > header (with a chunk-size of zero) on requests that have a body semantic > (POST, PUT). > As the code currently stands written, if we manually set `Content-Length: 0` > a ProtocolException is thrown > [RequestContent.java:106|https://github.com/apache/httpcomponents-core/blob/e3c770b55602eb9e5a45dfe7c6a07a1adede2c95/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestContent.java#L106] > and if we call the constructor with overwrite, this header is removed anyway > without being replaced. > The previous behaviour (4.x) is documented > [RequestContent.java:102|https://github.com/apache/httpcomponents-core/blob/a5c117028b7c620974304636d52f06f172f1d08b/httpcore/src/main/java/org/apache/http/protocol/RequestContent.java#L102] > > A suggested fix: re-add the null check and set Content-Length to zero. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCORE-759) Content-Length missing on null entity request content
[ https://issues.apache.org/jira/browse/HTTPCORE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770393#comment-17770393 ] Arturo Bernal edited comment on HTTPCORE-759 at 9/29/23 11:35 AM: -- I agree with your [~michael-o] , in the interpretation of RFC 9110. It clearly states that a 'Content-Length' header should not be sent when the request doesn't contain content and the method doesn't anticipate such data. was (Author: abernal): I agree with your [~michael-o] , interpretation of RFC 9110. It clearly states that a 'Content-Length' header should not be sent when the request doesn't contain content and the method doesn't anticipate such data. > Content-Length missing on null entity request content > - > > Key: HTTPCORE-759 > URL: https://issues.apache.org/jira/browse/HTTPCORE-759 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.2 >Reporter: Billy Jaime Beltran >Priority: Minor > Labels: easyfix > Time Spent: 10m > Remaining Estimate: 0h > > When making a POST request with an empty body, the following difference in > behaviour was observed. > In HttpComponents 4.4.x, a null body causes a Content-Length zero header to > be added. > In HttpComponents 5.2.x, a null body causes no Content-Length headers being > added. > While upgrading an Apache Camel application, we noticed that this breaks > calls to an upstream IIS server which replies with 411 (Length Required). > That server expects either a Content-Length 0 or a Transfer-Encoding: chunked > header (with a chunk-size of zero) on requests that have a body semantic > (POST, PUT). > As the code currently stands written, if we manually set `Content-Length: 0` > a ProtocolException is thrown > [RequestContent.java:106|https://github.com/apache/httpcomponents-core/blob/e3c770b55602eb9e5a45dfe7c6a07a1adede2c95/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestContent.java#L106] > and if we call the constructor with overwrite, this header is removed anyway > without being replaced. > The previous behaviour (4.x) is documented > [RequestContent.java:102|https://github.com/apache/httpcomponents-core/blob/a5c117028b7c620974304636d52f06f172f1d08b/httpcore/src/main/java/org/apache/http/protocol/RequestContent.java#L102] > > A suggested fix: re-add the null check and set Content-Length to zero. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-759) Content-Length missing on null entity request content
[ https://issues.apache.org/jira/browse/HTTPCORE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17770393#comment-17770393 ] Arturo Bernal commented on HTTPCORE-759: I agree with your [~michael-o] , interpretation of RFC 9110. It clearly states that a 'Content-Length' header should not be sent when the request doesn't contain content and the method doesn't anticipate such data. > Content-Length missing on null entity request content > - > > Key: HTTPCORE-759 > URL: https://issues.apache.org/jira/browse/HTTPCORE-759 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 5.2.2 >Reporter: Billy Jaime Beltran >Priority: Minor > Labels: easyfix > Time Spent: 10m > Remaining Estimate: 0h > > When making a POST request with an empty body, the following difference in > behaviour was observed. > In HttpComponents 4.4.x, a null body causes a Content-Length zero header to > be added. > In HttpComponents 5.2.x, a null body causes no Content-Length headers being > added. > While upgrading an Apache Camel application, we noticed that this breaks > calls to an upstream IIS server which replies with 411 (Length Required). > That server expects either a Content-Length 0 or a Transfer-Encoding: chunked > header (with a chunk-size of zero) on requests that have a body semantic > (POST, PUT). > As the code currently stands written, if we manually set `Content-Length: 0` > a ProtocolException is thrown > [RequestContent.java:106|https://github.com/apache/httpcomponents-core/blob/e3c770b55602eb9e5a45dfe7c6a07a1adede2c95/httpcore5/src/main/java/org/apache/hc/core5/http/protocol/RequestContent.java#L106] > and if we call the constructor with overwrite, this header is removed anyway > without being replaced. > The previous behaviour (4.x) is documented > [RequestContent.java:102|https://github.com/apache/httpcomponents-core/blob/a5c117028b7c620974304636d52f06f172f1d08b/httpcore/src/main/java/org/apache/http/protocol/RequestContent.java#L102] > > A suggested fix: re-add the null check and set Content-Length to zero. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17768779#comment-17768779 ] Arturo Bernal commented on HTTPCORE-756: HI [~reschke] Where is the commitment to the spirit of RFCs, which serve as strict standards designed to ensure seamless interoperability, consistency, and reliability across different systems? These standards are not mere suggestions; they are rigorously scrutinized guidelines agreed upon by experts. Deviating from them for 'flexibility' risks undermining their very purpose and can lead to inconsistencies that make systems less interoperable and reliable. In light of the library's strict adherence to RFCs, test cases would now need to be adapted to use httpcomponent as a 'validator' rather than a generator of potentially non-conforming messages. This shift could actually enhance the quality of conformance testing by ensuring it aligns with the latest standards Arturo > HttpCore HTTP Semantics conformance to RFC 9110 > --- > > Key: HTTPCORE-756 > URL: https://issues.apache.org/jira/browse/HTTPCORE-756 > Project: HttpComponents HttpCore > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > Ensure HttpCore conforms to RFC 9110. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2297) Documenting HttpClient 4.x status
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2297: -- Fix Version/s: 5.2.2 > Documenting HttpClient 4.x status > - > > Key: HTTPCLIENT-2297 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2297 > Project: HttpComponents HttpClient > Issue Type: Bug > Components: Site >Reporter: Henri Yandell >Priority: Major > Fix For: 5.2.2 > > > Hi HttpClient folk, > I noticed on > [https://github.com/apache/httpcomponents-core/pull/416#issuecomment-1688580065] > that 4.x is only receiving critical security and protocol fixes. I can see > on the final paragraph of the homepage ( > [https://hc.apache.org|https://hc.apache.org/] ) that 3.x is EOL, but I'm not > seeing any guidance on 4.x being on a deprecation path with limited updates. > Is a similar notice needed on the page now for 4.x? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2297) Documenting HttpClient 4.x status
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2297: -- Fix Version/s: 5.3-alpha2 > Documenting HttpClient 4.x status > - > > Key: HTTPCLIENT-2297 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2297 > Project: HttpComponents HttpClient > Issue Type: Bug > Components: Site >Reporter: Henri Yandell >Priority: Major > Fix For: 5.2.2, 5.3-alpha2 > > > Hi HttpClient folk, > I noticed on > [https://github.com/apache/httpcomponents-core/pull/416#issuecomment-1688580065] > that 4.x is only receiving critical security and protocol fixes. I can see > on the final paragraph of the homepage ( > [https://hc.apache.org|https://hc.apache.org/] ) that 3.x is EOL, but I'm not > seeing any guidance on 4.x being on a deprecation path with limited updates. > Is a similar notice needed on the page now for 4.x? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2297) Documenting HttpClient 4.x status
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2297: -- Component/s: Site > Documenting HttpClient 4.x status > - > > Key: HTTPCLIENT-2297 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2297 > Project: HttpComponents HttpClient > Issue Type: Bug > Components: Site >Reporter: Henri Yandell >Priority: Major > > Hi HttpClient folk, > I noticed on > [https://github.com/apache/httpcomponents-core/pull/416#issuecomment-1688580065] > that 4.x is only receiving critical security and protocol fixes. I can see > on the final paragraph of the homepage ( > [https://hc.apache.org|https://hc.apache.org/] ) that 3.x is EOL, but I'm not > seeing any guidance on 4.x being on a deprecation path with limited updates. > Is a similar notice needed on the page now for 4.x? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2297) Documenting HttpClient 4.x status
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal resolved HTTPCLIENT-2297. --- Resolution: Fixed merge in master > Documenting HttpClient 4.x status > - > > Key: HTTPCLIENT-2297 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2297 > Project: HttpComponents HttpClient > Issue Type: Bug >Reporter: Henri Yandell >Priority: Major > > Hi HttpClient folk, > I noticed on > [https://github.com/apache/httpcomponents-core/pull/416#issuecomment-1688580065] > that 4.x is only receiving critical security and protocol fixes. I can see > on the final paragraph of the homepage ( > [https://hc.apache.org|https://hc.apache.org/] ) that 3.x is EOL, but I'm not > seeing any guidance on 4.x being on a deprecation path with limited updates. > Is a similar notice needed on the page now for 4.x? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766913#comment-17766913 ] Arturo Bernal commented on HTTPCLIENT-2277: --- [~olegk] I'll review all the pending compliance points in RFC 9111 to ensure our implementation aligns fully. Arturo > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > Time Spent: 14h > Remaining Estimate: 0h > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2293) HttpClient HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764814#comment-17764814 ] Arturo Bernal commented on HTTPCLIENT-2293: --- Solved. > HttpClient HTTP Semantics conformance to RFC 9110 > - > > Key: HTTPCLIENT-2293 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2293 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761623#comment-17761623 ] Arturo Bernal commented on HTTPCORE-756: Julian, Your contributions to the RFC are noted. However, the primary objective here is to ensure our library's compliance with the latest standards. While I understand the importance of flexibility for conformance testing, I believe that adhering to the RFC should be our priority. Regards, > HttpCore HTTP Semantics conformance to RFC 9110 > --- > > Key: HTTPCORE-756 > URL: https://issues.apache.org/jira/browse/HTTPCORE-756 > Project: HttpComponents HttpCore > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > Time Spent: 2h 40m > Remaining Estimate: 0h > > Ensure HttpCore conforms to RFC 9110. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761488#comment-17761488 ] Arturo Bernal edited comment on HTTPCORE-756 at 9/2/23 5:07 PM: [~reschke] Could you elaborate on why we shouldn't align the project with the latest RFC? I understand the importance of retaining flexibility, especially for scenarios like conformance testing. I had assumed that adhering to the latest standards would be a priority. Would you suggest we introduce a mode or configuration that allows for non-standard behavior? This way, while the default behavior adheres to the RFC, users can override it when necessary for specific use cases. was (Author: abernal): [~reschke] Could you elaborate on why we shouldn't align the project with the latest RFC? I understand the importance of flexibility for scenarios like conformance testing, but I assumed that adhering to the latest standards would be a priority. I understand the importance of retaining flexibility, especially for scenarios like conformance testing. Would you suggest we provide a mode or configuration that allows for non-standard behavior, ensuring that while the default behavior adheres to the RFC, users can override it when necessary for specific use cases? > HttpCore HTTP Semantics conformance to RFC 9110 > --- > > Key: HTTPCORE-756 > URL: https://issues.apache.org/jira/browse/HTTPCORE-756 > Project: HttpComponents HttpCore > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > Time Spent: 2h 40m > Remaining Estimate: 0h > > Ensure HttpCore conforms to RFC 9110. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761488#comment-17761488 ] Arturo Bernal commented on HTTPCORE-756: [~reschke] Could you elaborate on why we shouldn't align the project with the latest RFC? I understand the importance of flexibility for scenarios like conformance testing, but I assumed that adhering to the latest standards would be a priority. I understand the importance of retaining flexibility, especially for scenarios like conformance testing. Would you suggest we provide a mode or configuration that allows for non-standard behavior, ensuring that while the default behavior adheres to the RFC, users can override it when necessary for specific use cases? > HttpCore HTTP Semantics conformance to RFC 9110 > --- > > Key: HTTPCORE-756 > URL: https://issues.apache.org/jira/browse/HTTPCORE-756 > Project: HttpComponents HttpCore > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > Time Spent: 2h 40m > Remaining Estimate: 0h > > Ensure HttpCore conforms to RFC 9110. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2293) HttpClient HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2293: -- Summary: HttpClient HTTP Semantics conformance to RFC 9110 (was: HttpCore HTTP Semantics conformance to RFC 9110) > HttpClient HTTP Semantics conformance to RFC 9110 > - > > Key: HTTPCLIENT-2293 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2293 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
[ https://issues.apache.org/jira/browse/HTTPCORE-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCORE-756: --- Description: Ensure HttpCore conforms to RFC 9110. > HttpCore HTTP Semantics conformance to RFC 9110 > --- > > Key: HTTPCORE-756 > URL: https://issues.apache.org/jira/browse/HTTPCORE-756 > Project: HttpComponents HttpCore > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > > Ensure HttpCore conforms to RFC 9110. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCORE-756) HttpCore HTTP Semantics conformance to RFC 9110
Arturo Bernal created HTTPCORE-756: -- Summary: HttpCore HTTP Semantics conformance to RFC 9110 Key: HTTPCORE-756 URL: https://issues.apache.org/jira/browse/HTTPCORE-756 Project: HttpComponents HttpCore Issue Type: Improvement Reporter: Arturo Bernal -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2293) HttpCore HTTP Semantics conformance to RFC 9110
Arturo Bernal created HTTPCLIENT-2293: - Summary: HttpCore HTTP Semantics conformance to RFC 9110 Key: HTTPCLIENT-2293 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2293 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCORE-746) Deadlock with Java Virtual Threads
[ https://issues.apache.org/jira/browse/HTTPCORE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753286#comment-17753286 ] Arturo Bernal edited comment on HTTPCORE-746 at 8/11/23 5:09 PM: - Merge in [5.3|https://github.com/apache/httpcomponents-core/tree/5.3.x] Please [~marsqing] check [https://github.com/apache/httpcomponents-core/pull/412] was (Author: abernal): Merge in 5.3 Please [~marsqing] check [https://github.com/apache/httpcomponents-core/pull/412] > Deadlock with Java Virtual Threads > -- > > Key: HTTPCORE-746 > URL: https://issues.apache.org/jira/browse/HTTPCORE-746 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 4.4.16 > Environment: openjdk version "20.0.1" 2023-04-18 > archlinux >Reporter: marsqing >Priority: Minor > Labels: volunteers-wanted > Fix For: Stuck > > Attachments: LoomWithApacheHttpClientTest.java > > Time Spent: 2.5h > Remaining Estimate: 0h > > [Virtual Threads|https://openjdk.org/jeps/444] is expected to be released in > JDK21 soon. Contrast to traditional 'platform threads', virtual threads will > usually unmount from platform threads when doing blocking operations. But > accoring to the doc above, when doing blocking operations in synchronized > block/method, the carrying platform thread will be pinned and can not carry > other virtual threads. > > This limitation will make the sample in the attachment deadlocked. Because > all the platform threads add pinned in > [await|https://github.com/apache/httpcomponents-core/blob/45d8a4cb391f591bb8d380b820437b651435774f/httpcore/src/main/java/org/apache/http/pool/AbstractConnPool.java#LL391C30-L391C30] > because it is in a [synchronized > block|https://github.com/apache/httpcomponents-core/blob/45d8a4cb391f591bb8d380b820437b651435774f/httpcore/src/main/java/org/apache/http/pool/AbstractConnPool.java#L244] > and no connection is available in the pool. The only virtual thread which > can release the connection to the pool is waiting for some platform threads > to carry it. That is the platform thread is waiting for a connection while > the connection is waiting the platform thread to carry its virtual thread and > return the connection. Thus a deadlock. > > If we replace the synchronized above with a ReentrantLock, no thead will be > pinned and no deadlock will occur. Accoring to the doc above, the limitation > may be removed in future JDK versions. But before that, we may need to > replace synchronized with some locks. Do we have any related plan? Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-746) Deadlock with Java Virtual Threads
[ https://issues.apache.org/jira/browse/HTTPCORE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17753286#comment-17753286 ] Arturo Bernal commented on HTTPCORE-746: Merge in 5.3 Please [~marsqing] check [https://github.com/apache/httpcomponents-core/pull/412] > Deadlock with Java Virtual Threads > -- > > Key: HTTPCORE-746 > URL: https://issues.apache.org/jira/browse/HTTPCORE-746 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 4.4.16 > Environment: openjdk version "20.0.1" 2023-04-18 > archlinux >Reporter: marsqing >Priority: Minor > Labels: volunteers-wanted > Fix For: Stuck > > Attachments: LoomWithApacheHttpClientTest.java > > Time Spent: 2.5h > Remaining Estimate: 0h > > [Virtual Threads|https://openjdk.org/jeps/444] is expected to be released in > JDK21 soon. Contrast to traditional 'platform threads', virtual threads will > usually unmount from platform threads when doing blocking operations. But > accoring to the doc above, when doing blocking operations in synchronized > block/method, the carrying platform thread will be pinned and can not carry > other virtual threads. > > This limitation will make the sample in the attachment deadlocked. Because > all the platform threads add pinned in > [await|https://github.com/apache/httpcomponents-core/blob/45d8a4cb391f591bb8d380b820437b651435774f/httpcore/src/main/java/org/apache/http/pool/AbstractConnPool.java#LL391C30-L391C30] > because it is in a [synchronized > block|https://github.com/apache/httpcomponents-core/blob/45d8a4cb391f591bb8d380b820437b651435774f/httpcore/src/main/java/org/apache/http/pool/AbstractConnPool.java#L244] > and no connection is available in the pool. The only virtual thread which > can release the connection to the pool is waiting for some platform threads > to carry it. That is the platform thread is waiting for a connection while > the connection is waiting the platform thread to carry its virtual thread and > return the connection. Thus a deadlock. > > If we replace the synchronized above with a ReentrantLock, no thead will be > pinned and no deadlock will occur. Accoring to the doc above, the limitation > may be removed in future JDK versions. But before that, we may need to > replace synchronized with some locks. Do we have any related plan? Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-746) Deadlock with Java Virtual Threads
[ https://issues.apache.org/jira/browse/HTTPCORE-746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal resolved HTTPCORE-746. Resolution: Fixed > Deadlock with Java Virtual Threads > -- > > Key: HTTPCORE-746 > URL: https://issues.apache.org/jira/browse/HTTPCORE-746 > Project: HttpComponents HttpCore > Issue Type: Improvement > Components: HttpCore >Affects Versions: 4.4.16 > Environment: openjdk version "20.0.1" 2023-04-18 > archlinux >Reporter: marsqing >Priority: Minor > Labels: volunteers-wanted > Fix For: Stuck > > Attachments: LoomWithApacheHttpClientTest.java > > Time Spent: 2.5h > Remaining Estimate: 0h > > [Virtual Threads|https://openjdk.org/jeps/444] is expected to be released in > JDK21 soon. Contrast to traditional 'platform threads', virtual threads will > usually unmount from platform threads when doing blocking operations. But > accoring to the doc above, when doing blocking operations in synchronized > block/method, the carrying platform thread will be pinned and can not carry > other virtual threads. > > This limitation will make the sample in the attachment deadlocked. Because > all the platform threads add pinned in > [await|https://github.com/apache/httpcomponents-core/blob/45d8a4cb391f591bb8d380b820437b651435774f/httpcore/src/main/java/org/apache/http/pool/AbstractConnPool.java#LL391C30-L391C30] > because it is in a [synchronized > block|https://github.com/apache/httpcomponents-core/blob/45d8a4cb391f591bb8d380b820437b651435774f/httpcore/src/main/java/org/apache/http/pool/AbstractConnPool.java#L244] > and no connection is available in the pool. The only virtual thread which > can release the connection to the pool is waiting for some platform threads > to carry it. That is the platform thread is waiting for a connection while > the connection is waiting the platform thread to carry its virtual thread and > return the connection. Thus a deadlock. > > If we replace the synchronized above with a ReentrantLock, no thead will be > pinned and no deadlock will occur. Accoring to the doc above, the limitation > may be removed in future JDK versions. But before that, we may need to > replace synchronized with some locks. Do we have any related plan? Thanks! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2290) Refactoring of synchronized sections to use lock primitives in light of virtual threads
Arturo Bernal created HTTPCLIENT-2290:
-
Summary: Refactoring of synchronized sections to use lock
primitives in light of virtual threads
Key: HTTPCLIENT-2290
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2290
Project: HttpComponents HttpClient
Issue Type: Improvement
Reporter: Arturo Bernal
With the upcoming inclusion of virtual threads in JDK 21, we need to review and
refactor sections of the HttpClient codebase that use the {{synchronized}}
keyword.
As virtual threads might introduce performance and deadlock issues with the
current {{synchronized}} sections, we'll move towards using explicit lock
primitives provided by the {{java.util.concurrent.locks}} package, such as
{{{}ReentrantLock{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCORE-747) Some flaky tests
[
https://issues.apache.org/jira/browse/HTTPCORE-747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal resolved HTTPCORE-747.
Fix Version/s: 5.2.3
Resolution: Fixed
[~agorguy]
Fix committed to master.
Arturo
> Some flaky tests
>
>
> Key: HTTPCORE-747
> URL: https://issues.apache.org/jira/browse/HTTPCORE-747
> Project: HttpComponents HttpCore
> Issue Type: Improvement
>Reporter: Agor
>Assignee: Arturo Bernal
>Priority: Minor
> Labels: volunteers-wanted
> Fix For: 5.2.3
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Hello,
> We tried running this project and discovered that it contains some flaky
> tests (i.e., tests that nondeterministically pass and fail). We found these
> tests to fail more frequently when running them on certain machines of ours.
> To prevent others from running this project and its tests in machines that
> may result in flaky tests, we suggest adding information to the README.md
> file indicating the minimum resource configuration for running the tests of
> this project as to prevent observation of test flakiness.
> If we run this project in a machine with 1cpu and 1gb ram, we observe flaky
> tests. We found that the tests in this project did not have any flaky tests
> when we ran it on machines with 2cpu and 2gb ram.
> Here is a list of the tests we have identified and their likelihood of
> failure on a system with less than the recommended 2 CPUs and 2 GB RAM.
> # org.apache.hc.core5.pool.TestPoolEntry#testExpiry
> # org.apache.hc.core5.pool.TestLaxConnPool#testCloseIdle
> #
> org.apache.hc.core5.testing.classic.ClassicIntegrationTest#testHttpPostsWithExpectContinue
> Please let me know if you would like us to create a pull request on this
> matter (possibly to the readme of this project).
> Thank you for your attention to this matter. We hope that our recommendations
> will be helpful in improving the quality and performance of your project,
> especially for others to use.
> h3. Reproducing
> {code:java}
> FROM maven:3.5.4-jdk-11
> WORKDIR /home/
> RUN git clone https://github.com/apache/httpcomponents-core && \
> cd httpcomponents-core && \
> git checkout 9f8cdb17c85defb267fc814d6094bfda3c8ff1a7
> WORKDIR /home/httpcomponents-core
> RUN mvn install -DskipTests
> ENTRYPOINT ["mvn", "test", "-fn"]
> {code}
> Build the image:
> {code:java}
> $> mkdir tmp
> $> cp Dockerfile tmp
> $> cd tmp
> $> docker build -t http-core . # estimated time of build 3m
> {code}
> Running:
> this configuration likely prevents flakiness (no flakiness in 10 runs)
> {code:java}
> $> docker run --rm --memory=2g --cpus=2 --memory-swap=-1 http-core | tee
> output.txt
> $> grep "Failures:" output.txt # checking results{code}
> this other configuration –similar to the previous– can’t prevent flaky tests
> (observation in 10 runs)
> {code:java}
> $> docker run --rm --memory=500m --cpus=1 --memory-swap=-1 http-core | tee
> output2.txt
> $> grep "Failures:" output2.txt # checking results{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Assigned] (HTTPCORE-747) Some flaky tests
[
https://issues.apache.org/jira/browse/HTTPCORE-747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal reassigned HTTPCORE-747:
--
Assignee: Arturo Bernal
> Some flaky tests
>
>
> Key: HTTPCORE-747
> URL: https://issues.apache.org/jira/browse/HTTPCORE-747
> Project: HttpComponents HttpCore
> Issue Type: Improvement
>Reporter: Agor
>Assignee: Arturo Bernal
>Priority: Minor
> Labels: volunteers-wanted
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Hello,
> We tried running this project and discovered that it contains some flaky
> tests (i.e., tests that nondeterministically pass and fail). We found these
> tests to fail more frequently when running them on certain machines of ours.
> To prevent others from running this project and its tests in machines that
> may result in flaky tests, we suggest adding information to the README.md
> file indicating the minimum resource configuration for running the tests of
> this project as to prevent observation of test flakiness.
> If we run this project in a machine with 1cpu and 1gb ram, we observe flaky
> tests. We found that the tests in this project did not have any flaky tests
> when we ran it on machines with 2cpu and 2gb ram.
> Here is a list of the tests we have identified and their likelihood of
> failure on a system with less than the recommended 2 CPUs and 2 GB RAM.
> # org.apache.hc.core5.pool.TestPoolEntry#testExpiry
> # org.apache.hc.core5.pool.TestLaxConnPool#testCloseIdle
> #
> org.apache.hc.core5.testing.classic.ClassicIntegrationTest#testHttpPostsWithExpectContinue
> Please let me know if you would like us to create a pull request on this
> matter (possibly to the readme of this project).
> Thank you for your attention to this matter. We hope that our recommendations
> will be helpful in improving the quality and performance of your project,
> especially for others to use.
> h3. Reproducing
> {code:java}
> FROM maven:3.5.4-jdk-11
> WORKDIR /home/
> RUN git clone https://github.com/apache/httpcomponents-core && \
> cd httpcomponents-core && \
> git checkout 9f8cdb17c85defb267fc814d6094bfda3c8ff1a7
> WORKDIR /home/httpcomponents-core
> RUN mvn install -DskipTests
> ENTRYPOINT ["mvn", "test", "-fn"]
> {code}
> Build the image:
> {code:java}
> $> mkdir tmp
> $> cp Dockerfile tmp
> $> cd tmp
> $> docker build -t http-core . # estimated time of build 3m
> {code}
> Running:
> this configuration likely prevents flakiness (no flakiness in 10 runs)
> {code:java}
> $> docker run --rm --memory=2g --cpus=2 --memory-swap=-1 http-core | tee
> output.txt
> $> grep "Failures:" output.txt # checking results{code}
> this other configuration –similar to the previous– can’t prevent flaky tests
> (observation in 10 runs)
> {code:java}
> $> docker run --rm --memory=500m --cpus=1 --memory-swap=-1 http-core | tee
> output2.txt
> $> grep "Failures:" output2.txt # checking results{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-750) H2ConnPool creates too many conns under high load at initialization time
[
https://issues.apache.org/jira/browse/HTTPCORE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728923#comment-17728923
]
Arturo Bernal commented on HTTPCORE-750:
Calling {{get(1, TimeUnit.MILLISECONDS)}} does not fundamentally change this.
It just reduces the potential blocking time to a very small amount. While this
might reduce the probability of experiencing the aforementioned issues, it does
not eliminate it. Moreover, if the operation takes longer than 1 millisecond to
complete, {{{}get(1, TimeUnit.MILLISECONDS){}}}will throw a
{{{}TimeoutException{}}}, which would need to be handled.
As for the {{sessionInProgress}} flag, you're correct that it would need to be
updated within the callback in a thread-safe manner. This is not fundamentally
different from other callback logic that updates shared state (such as
{{{}poolEntry.session{}}}), and it does not inherently prevent the solution
from working. Of course this is just an idea
> H2ConnPool creates too many conns under high load at initialization time
>
>
> Key: HTTPCORE-750
> URL: https://issues.apache.org/jira/browse/HTTPCORE-750
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.2.1
> Environment: Linux / Ubuntu / openjdk version "1.8.0_292"
>Reporter: Gaojie Liu
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We are using httpcore5 5.2.1 and we observed that at application startup
> time, httpclient5 could create more than 1 conn per endpoint and in some
> cases, too many connections caused OOM issue since httpclient5 creates a pair
> of input/output buffer per conn.
> The regression is introduced by HTTPCORE-750, which made this change:
> [https://github.com/apache/httpcomponents-core/commit/14caf43eae407c544161c7f92329e8beb42a3534]
>
> {code:java}
> if (poolEntry.session != null) {
> callback.completed(poolEntry.session);
> } else {
> poolEntry.requestQueue.add(callback);
> if (poolEntry.sessionFuture != null &&
> poolEntry.sessionFuture.isDone()) {
> poolEntry.sessionFuture = null;
> } {code}
> When poolEntry.sessionFuture.isDone() is true, the connection is already
> ready, but the existing logic will abandon it and create a new one, and under
> high load, this logic could create a lot of conns per endpoint, which
> consumes a lot of memory.
>
> The proposed fix:
> {code:java}
> if (poolEntry.session != null) {
> callback.completed(poolEntry.session);
> } else {
> poolEntry.requestQueue.add(callback);
> if (poolEntry.sessionFuture != null &&
> poolEntry.sessionFuture.isDone()) {
> // Check whether we should recreate a new conn or not
> try {
> poolEntry.session = poolEntry.sessionFuture.get();
> while (true) {
> final FutureCallback pendingCallback =
> poolEntry.requestQueue.poll();
> if (pendingCallback != null) {
> pendingCallback.completed(poolEntry.session);
> } else {
> break;
> }
> }
> } catch (final Exception e) {
> poolEntry.sessionFuture = null;
> }
> } {code}
> I am planning to cut a PR for this.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-750) H2ConnPool creates too many conns under high load at initialization time
[
https://issues.apache.org/jira/browse/HTTPCORE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728852#comment-17728852
]
Arturo Bernal commented on HTTPCORE-750:
Maybe Instead of modifying {{BasicFuture}} or relying on {{{}Future.get(){}}},
we can add an additional boolean flag, {{{}sessionInProgress{}}}, to
{{{}PoolEntry{}}}. This flag would indicate whether a session creation process
is currently in progress.
You could adjust the logic so that a new session will only be created if
{{poolEntry.sessionFuture}} is null and {{poolEntry.sessionInProgress}} is
false. Upon completion, failure, or cancellation of the session creation, we
would set {{sessionInProgress}} back to false.
> H2ConnPool creates too many conns under high load at initialization time
>
>
> Key: HTTPCORE-750
> URL: https://issues.apache.org/jira/browse/HTTPCORE-750
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
>Affects Versions: 5.2.1
> Environment: Linux / Ubuntu / openjdk version "1.8.0_292"
>Reporter: Gaojie Liu
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We are using httpcore5 5.2.1 and we observed that at application startup
> time, httpclient5 could create more than 1 conn per endpoint and in some
> cases, too many connections caused OOM issue since httpclient5 creates a pair
> of input/output buffer per conn.
> The regression is introduced by HTTPCORE-750, which made this change:
> [https://github.com/apache/httpcomponents-core/commit/14caf43eae407c544161c7f92329e8beb42a3534]
>
> {code:java}
> if (poolEntry.session != null) {
> callback.completed(poolEntry.session);
> } else {
> poolEntry.requestQueue.add(callback);
> if (poolEntry.sessionFuture != null &&
> poolEntry.sessionFuture.isDone()) {
> poolEntry.sessionFuture = null;
> } {code}
> When poolEntry.sessionFuture.isDone() is true, the connection is already
> ready, but the existing logic will abandon it and create a new one, and under
> high load, this logic could create a lot of conns per endpoint, which
> consumes a lot of memory.
>
> The proposed fix:
> {code:java}
> if (poolEntry.session != null) {
> callback.completed(poolEntry.session);
> } else {
> poolEntry.requestQueue.add(callback);
> if (poolEntry.sessionFuture != null &&
> poolEntry.sessionFuture.isDone()) {
> // Check whether we should recreate a new conn or not
> try {
> poolEntry.session = poolEntry.sessionFuture.get();
> while (true) {
> final FutureCallback pendingCallback =
> poolEntry.requestQueue.poll();
> if (pendingCallback != null) {
> pendingCallback.completed(poolEntry.session);
> } else {
> break;
> }
> }
> } catch (final Exception e) {
> poolEntry.sessionFuture = null;
> }
> } {code}
> I am planning to cut a PR for this.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728135#comment-17728135 ] Arturo Bernal commented on HTTPCLIENT-2277: --- AFAIK the Apache HttpComponents library does not inherently support storing incomplete responses or combining partial content in the caching mechanism, h3. [3.3. |https://www.rfc-editor.org/rfc/rfc9111#section-3.3] [Storing Incomplete Responses|https://www.rfc-editor.org/rfc/rfc9111#name-storing-incomplete-response] * *Storing Incomplete Responses:* Apache HttpClient, at that time, didn't have built-in mechanisms to identify and store incomplete responses. In general, if an HTTP response is received and it is not flagged with an error status (like HTTP 206 Partial Content), it is treated as a complete response h3. [3.4. |https://www.rfc-editor.org/rfc/rfc9111#section-3.4] [Combining Partial Content|https://www.rfc-editor.org/rfc/rfc9111#name-combining-partial-content] * *Combining Partial Content:* Similarly, Apache HttpClient doesn't inherently support the combining of partial content. The library is typically used to issue HTTP requests and process HTTP responses. How partial content is handled is usually a higher-level concern, not something that HttpClient manages itself. > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2279) Improve Handling of HTTP Conditional Requests, Status Codes and Weak ETags
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17728111#comment-17728111 ] Arturo Bernal commented on HTTPCLIENT-2279: --- [~olegk] Fair enough . > Improve Handling of HTTP Conditional Requests, Status Codes and Weak ETags > -- > > Key: HTTPCLIENT-2279 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2279 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > > Our system currently does not fully support the handling of HTTP conditional > requests and the appropriate status codes as outlined in the HTTP/1.1 > specification RFC 9110. There are three main issues to be addressed: > * > Lack of support for the HTTP 412 (Precondition Failed) status code, which > indicates that one or more conditions given in the request header fields > evaluated to false when tested on the server. > * Our system may not check all header fields marked as If-Match or > If-None-Match. According to the specification, all instances of these headers > should be considered. > * The system might not properly handle cases where a weak ETag is used with > a range request. > By addressing these issues, we can improve the robustness of our system and > its compliance with the HTTP specification. This will ensure correct and > consistent behavior, particularly when dealing with conditional requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2279) Improve Handling of HTTP Conditional Requests, Status Codes and Weak ETags
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17727623#comment-17727623 ] Arturo Bernal commented on HTTPCLIENT-2279: --- [~olegk] Yes, it somewhat ties into the issue/improvement related to RFC9110. I created a ticket because I was already working on this prior to HTTPCLIENT-2277, and I had already completed half of the solution. In any case I can continue with HTTPCLIENT-2277 and leave as a draft this one. > Improve Handling of HTTP Conditional Requests, Status Codes and Weak ETags > -- > > Key: HTTPCLIENT-2279 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2279 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > > Our system currently does not fully support the handling of HTTP conditional > requests and the appropriate status codes as outlined in the HTTP/1.1 > specification RFC 9110. There are three main issues to be addressed: > * > Lack of support for the HTTP 412 (Precondition Failed) status code, which > indicates that one or more conditions given in the request header fields > evaluated to false when tested on the server. > * Our system may not check all header fields marked as If-Match or > If-None-Match. According to the specification, all instances of these headers > should be considered. > * The system might not properly handle cases where a weak ETag is used with > a range request. > By addressing these issues, we can improve the robustness of our system and > its compliance with the HTTP specification. This will ensure correct and > consistent behavior, particularly when dealing with conditional requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2279) Improve Handling of HTTP Conditional Requests, Status Codes and Weak ETags
Arturo Bernal created HTTPCLIENT-2279: - Summary: Improve Handling of HTTP Conditional Requests, Status Codes and Weak ETags Key: HTTPCLIENT-2279 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2279 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal Our system currently does not fully support the handling of HTTP conditional requests and the appropriate status codes as outlined in the HTTP/1.1 specification RFC 9110. There are three main issues to be addressed: * Lack of support for the HTTP 412 (Precondition Failed) status code, which indicates that one or more conditions given in the request header fields evaluated to false when tested on the server. * Our system may not check all header fields marked as If-Match or If-None-Match. According to the specification, all instances of these headers should be considered. * The system might not properly handle cases where a weak ETag is used with a range request. By addressing these issues, we can improve the robustness of our system and its compliance with the HTTP specification. This will ensure correct and consistent behavior, particularly when dealing with conditional requests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726967#comment-17726967 ] Arturo Bernal commented on HTTPCLIENT-2277: --- [~olegk] Wow, you really dropped a bombshell there. It looks like we've got a behemoth of a task on our hands with the cache structure. Looks like we're in for quite a ride with this one! > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726381#comment-17726381 ] Arturo Bernal commented on HTTPCLIENT-2277: --- 4.2.1. Calculating Freshness Lifetime * If the cache is shared and the s-maxage response directive ([Section 5.2.2.10|https://httpwg.org/specs/rfc9111.html#cache-response-directive.s-maxage]) is present, use its value, or It's apparent we need to review our handling of instances when multiple values exist for a given directive, such as multiple Cache-Control: max-age directives or two Expires header field lines. We should employ a meticulous approach: utilize the first occurrence, or in the absence of a clear choice, consider the response as stale * When there is more than one value present for a given directive (e.g., two [Expires|https://httpwg.org/specs/rfc9111.html#field.expires] header field lines or multiple Cache-Control: max-age directives), either the first occurrence should be used or the response should be considered stale. If directives conflict (e.g., both max-age and no-cache are present), the most restrictive directive should be honored. Caches are encouraged to consider responses that have invalid freshness information (e.g., a max-age directive with non-integer content) to be stale. > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725884#comment-17725884 ] Arturo Bernal commented on HTTPCLIENT-2277: --- RFC 9111, Section 5.2.3 - Extension Directives: The Cache-Control header field can be expanded through custom extension cache directives. At present, our cache implementation does not support custom directives > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725554#comment-17725554 ] Arturo Bernal commented on HTTPCLIENT-2277: --- RFC 9111, section 4.1 * When a cache receives a request that can be satisfied by a stored response and that stored response contains a [Vary|https://httpwg.org/specs/rfc9110.html#field.vary] header field ([Section 12.5.5|https://httpwg.org/specs/rfc9110.html#field.vary] of [??[HTTP]??|https://httpwg.org/specs/rfc9111.html#HTTP]), the cache _MUST NOT_ use that stored response without revalidation unless all the presented request header fields nominated by that Vary field value match those fields in the original request (i.e., the request that caused the cached response to be stored). Under our existing system architecture, we have yet to implement the Vary header validation as specified in these requirements > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724779#comment-17724779 ] Arturo Bernal commented on HTTPCLIENT-2277: --- "RFC 9111, section 5.5 The 'Warning' header field has been obsoleted by RFC 9111. Previously, this field was used to carry additional information about the status or transformation of a message. Now, this information should be gleaned from examining other header fields, such as 'Age'. At the moment, the HTTP cache uses the 'Warning' header to determine if a response is stale. > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724708#comment-17724708 ] Arturo Bernal commented on HTTPCLIENT-2277: --- RFC 9111, section 5.2.2.3 A 'must-understand' cache directive has been introduced. Caches are no longer required to understand the semantics of new response status codes unless the 'must-understand' directive is present. it's unclear whether this change is necessary in the current implementation, as it depends on how the cache currently handles response status codes?? > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2277) HttpClient Cache conformance to RFC 9111
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724697#comment-17724697 ] Arturo Bernal commented on HTTPCLIENT-2277: --- "RFC 9111, section 5.1 The 'Age' response header field indicates the sender's estimate of the time since the response was generated or validated at the origin server. When multiple Age headers are present, the first one should be used and the rest discarded. If the Age header value is invalid (not a non-negative integer), the cache should ignore it. At the moment, the HTTP cache returns false and considers the response not cacheable when multiple Age headers are present. This behavior needs to be updated to align with RFC 9111, which requires the first Age header to be used and the rest discarded rather than considering the response not cacheable. Also HTTP cache uses the largest Age value when multiple Age headers are present and replaces invalid Age values with a default maximum age > HttpClient Cache conformance to RFC 9111 > > > Key: HTTPCLIENT-2277 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2277 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Reporter: Oleg Kalnichevski >Priority: Major > > Ensure HttpClient Cache conforms to RFC 9111. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Resolved] (HTTPCLIENT-2276) Implementing 'Calculating Secondary Keys with Vary' as Per RFC 7234 for HTTP Cache Validity"
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal resolved HTTPCLIENT-2276. --- Resolution: Won't Fix In the RFC 9111 supersedes certain functionalities from RFC 7234, including the "Calculating Secondary Keys with Vary" method. Given that this specific feature is no longer required in the new standard, it doesn't make sense to implement it now, only to remove it later. > Implementing 'Calculating Secondary Keys with Vary' as Per RFC 7234 for HTTP > Cache Validity" > > > Key: HTTPCLIENT-2276 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2276 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > > The purpose of this ticket is to implement the method "Calculating Secondary > Keys with Vary" as described in RFC 7234, section 4.1 for HTTP Cache Validity > in our caching solution. > Currently, our solution doesn't consider the rules for matching the selecting > header fields mentioned in the RFC while validating cache. The requirement is > to match the selecting header fields from the original request and the > presented request by applying the following transformations: > # Adding or removing whitespace, where allowed in the header field's syntax. > # Combining multiple header fields with the same field name. > # Normalizing both header field values in a way that is known to have > identical semantics, according to the header field's specification (e.g., > reordering field values when order is not significant; case-normalization, > where values are defined to be case-insensitive). > The ticket includes handling of cases where a header field is absent from a > request, it should match another request only if it is also absent there. > Also, a Vary header field-value of "*" should always fail to match. > Lastly, the implementation should handle the selection of response when > multiple selected responses are available, and the case when no selected > response is available. > This enhancement is expected to improve our cache validity check and adhere > to the standard protocol as outlined in RFC 7234. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2276) Implementing 'Calculating Secondary Keys with Vary' as Per RFC 7234 for HTTP Cache Validity"
Arturo Bernal created HTTPCLIENT-2276: - Summary: Implementing 'Calculating Secondary Keys with Vary' as Per RFC 7234 for HTTP Cache Validity" Key: HTTPCLIENT-2276 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2276 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal The purpose of this ticket is to implement the method "Calculating Secondary Keys with Vary" as described in RFC 7234, section 4.1 for HTTP Cache Validity in our caching solution. Currently, our solution doesn't consider the rules for matching the selecting header fields mentioned in the RFC while validating cache. The requirement is to match the selecting header fields from the original request and the presented request by applying the following transformations: # Adding or removing whitespace, where allowed in the header field's syntax. # Combining multiple header fields with the same field name. # Normalizing both header field values in a way that is known to have identical semantics, according to the header field's specification (e.g., reordering field values when order is not significant; case-normalization, where values are defined to be case-insensitive). The ticket includes handling of cases where a header field is absent from a request, it should match another request only if it is also absent there. Also, a Vary header field-value of "*" should always fail to match. Lastly, the implementation should handle the selection of response when multiple selected responses are available, and the case when no selected response is available. This enhancement is expected to improve our cache validity check and adhere to the standard protocol as outlined in RFC 7234. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-1347) gzip responses doubly cached
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723732#comment-17723732
]
Arturo Bernal edited comment on HTTPCLIENT-1347 at 5/17/23 8:45 PM:
I am writing to provide an update on the testing conducted for Apache
HttpComponents 5.3. I have run multiple tests to verify the functionality of
the new version and can confirm that it is operating as expected.
The test involved setting up a local HTTP server using the HttpComponents and
serving static files from a specified directory. The server correctly processed
GET requests and served the requested files with appropriate cache-control
headers, thus enabling client-side caching.
Repeated requests for the same file were served from the cache when valid,
which is a clear indication that the caching mechanism is working as expected.
The server also correctly processed different paths and served the
corresponding files.
Here is a summary of the test results:
# GET request to {{{}[http://localhost:8000/assert/1]{}}}: Successful.
Response served with cache-control headers.
# GET request to {{{}[http://localhost:8000/assert/2]{}}}: Successful.
Response served with cache-control headers.
# Subsequent GET requests to the above URLs were served from the cache, as
intended.
you can see the detail here [^output.out]
Based on these test results, I believe that Apache HttpComponents 5.3 is
functioning correctly and is ready for further use or deployment. Or maybe I'm
missing something.
In order to facilitate replication of these tests, I have attached a .zip file
containing the necessary classes to set up and run the local server in the
Apache HttpComponents environment. You can use these classes to perform the
same tests that I did.
Please let me know if you require additional information or further testing.
was (Author: arturobernalg):
I am writing to provide an update on the testing conducted for Apache
HttpComponents 5.3. I have run multiple tests to verify the functionality of
the new version and can confirm that it is operating as expected.
The test involved setting up a local HTTP server using the HttpComponents and
serving static files from a specified directory. The server correctly processed
GET requests and served the requested files with appropriate cache-control
headers, thus enabling client-side caching.
Repeated requests for the same file were served from the cache when valid,
which is a clear indication that the caching mechanism is working as expected.
The server also correctly processed different paths and served the
corresponding files.
Here is a summary of the test results:
# GET request to {{{}http://localhost:8000/assert/1{}}}: Successful. Response
served with cache-control headers.
# GET request to {{{}http://localhost:8000/assert/2{}}}: Successful. Response
served with cache-control headers.
# Subsequent GET requests to the above URLs were served from the cache, as
intended.
you can see the detail here [^output.out]
Based on these test results, I believe that Apache HttpComponents 5.3 is
functioning correctly and is ready for further use or deployment.
In order to facilitate replication of these tests, I have attached a .zip file
containing the necessary classes to set up and run the local server in the
Apache HttpComponents environment. You can use these classes to perform the
same tests that I did.
Please let me know if you require additional information or further testing.
> gzip responses doubly cached
>
>
> Key: HTTPCLIENT-1347
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpCache
>Affects Versions: 4.2.5
> Environment: ARCH Linux kernel 3.8.8-1
> node.js 0.8.22
>Reporter: Adam Patacchiola
>Priority: Major
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
> Attachments: Archive.zip, Screen Shot 2014-01-11 at 7.11.36 PM.png,
> Screen Shot 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png,
> httpClientCacheTest.tar.gz, httpClientTestServer.js, output.out
>
>
> Compressed responses are cached twice.
> Run the attached server (node.js 0.8.22) and client tests. Create an "assets"
> directory under where you are running the server and add two files named 1
> and 2 ( < 100 bytes) . You will see that after the test is run the cache
> dump output displays 2 sets of entries for each request, each containing the
> full content length of the file.
> Changing the implementation of HttpCacheStorage updateEntry to not update non
> existent entries (as I believe the correct implementation should do) throws
> exceptions.
--
This message was sent by Atlassian Jira
[jira] [Commented] (HTTPCLIENT-1347) gzip responses doubly cached
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17723732#comment-17723732
]
Arturo Bernal commented on HTTPCLIENT-1347:
---
I am writing to provide an update on the testing conducted for Apache
HttpComponents 5.3. I have run multiple tests to verify the functionality of
the new version and can confirm that it is operating as expected.
The test involved setting up a local HTTP server using the HttpComponents and
serving static files from a specified directory. The server correctly processed
GET requests and served the requested files with appropriate cache-control
headers, thus enabling client-side caching.
Repeated requests for the same file were served from the cache when valid,
which is a clear indication that the caching mechanism is working as expected.
The server also correctly processed different paths and served the
corresponding files.
Here is a summary of the test results:
# GET request to {{{}http://localhost:8000/assert/1{}}}: Successful. Response
served with cache-control headers.
# GET request to {{{}http://localhost:8000/assert/2{}}}: Successful. Response
served with cache-control headers.
# Subsequent GET requests to the above URLs were served from the cache, as
intended.
you can see the detail here [^output.out]
Based on these test results, I believe that Apache HttpComponents 5.3 is
functioning correctly and is ready for further use or deployment.
In order to facilitate replication of these tests, I have attached a .zip file
containing the necessary classes to set up and run the local server in the
Apache HttpComponents environment. You can use these classes to perform the
same tests that I did.
Please let me know if you require additional information or further testing.
> gzip responses doubly cached
>
>
> Key: HTTPCLIENT-1347
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpCache
>Affects Versions: 4.2.5
> Environment: ARCH Linux kernel 3.8.8-1
> node.js 0.8.22
>Reporter: Adam Patacchiola
>Priority: Major
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
> Attachments: Archive.zip, Screen Shot 2014-01-11 at 7.11.36 PM.png,
> Screen Shot 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png,
> httpClientCacheTest.tar.gz, httpClientTestServer.js, output.out
>
>
> Compressed responses are cached twice.
> Run the attached server (node.js 0.8.22) and client tests. Create an "assets"
> directory under where you are running the server and add two files named 1
> and 2 ( < 100 bytes) . You will see that after the test is run the cache
> dump output displays 2 sets of entries for each request, each containing the
> full content length of the file.
> Changing the implementation of HttpCacheStorage updateEntry to not update non
> existent entries (as I believe the correct implementation should do) throws
> exceptions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-1347) gzip responses doubly cached
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-1347: -- Attachment: output.out > gzip responses doubly cached > > > Key: HTTPCLIENT-1347 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.5 > Environment: ARCH Linux kernel 3.8.8-1 > node.js 0.8.22 >Reporter: Adam Patacchiola >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > Attachments: Archive.zip, Screen Shot 2014-01-11 at 7.11.36 PM.png, > Screen Shot 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png, > httpClientCacheTest.tar.gz, httpClientTestServer.js, output.out > > > Compressed responses are cached twice. > Run the attached server (node.js 0.8.22) and client tests. Create an "assets" > directory under where you are running the server and add two files named 1 > and 2 ( < 100 bytes) . You will see that after the test is run the cache > dump output displays 2 sets of entries for each request, each containing the > full content length of the file. > Changing the implementation of HttpCacheStorage updateEntry to not update non > existent entries (as I believe the correct implementation should do) throws > exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-1347) gzip responses doubly cached
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-1347: -- Attachment: (was: output.out) > gzip responses doubly cached > > > Key: HTTPCLIENT-1347 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.5 > Environment: ARCH Linux kernel 3.8.8-1 > node.js 0.8.22 >Reporter: Adam Patacchiola >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > Attachments: Archive.zip, Screen Shot 2014-01-11 at 7.11.36 PM.png, > Screen Shot 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png, > httpClientCacheTest.tar.gz, httpClientTestServer.js, output.out > > > Compressed responses are cached twice. > Run the attached server (node.js 0.8.22) and client tests. Create an "assets" > directory under where you are running the server and add two files named 1 > and 2 ( < 100 bytes) . You will see that after the test is run the cache > dump output displays 2 sets of entries for each request, each containing the > full content length of the file. > Changing the implementation of HttpCacheStorage updateEntry to not update non > existent entries (as I believe the correct implementation should do) throws > exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-1347) gzip responses doubly cached
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-1347: -- Attachment: output.out > gzip responses doubly cached > > > Key: HTTPCLIENT-1347 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.5 > Environment: ARCH Linux kernel 3.8.8-1 > node.js 0.8.22 >Reporter: Adam Patacchiola >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > Attachments: Archive.zip, Screen Shot 2014-01-11 at 7.11.36 PM.png, > Screen Shot 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png, > httpClientCacheTest.tar.gz, httpClientTestServer.js, output.out > > > Compressed responses are cached twice. > Run the attached server (node.js 0.8.22) and client tests. Create an "assets" > directory under where you are running the server and add two files named 1 > and 2 ( < 100 bytes) . You will see that after the test is run the cache > dump output displays 2 sets of entries for each request, each containing the > full content length of the file. > Changing the implementation of HttpCacheStorage updateEntry to not update non > existent entries (as I believe the correct implementation should do) throws > exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-1347) gzip responses doubly cached
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-1347: -- Attachment: Archive.zip > gzip responses doubly cached > > > Key: HTTPCLIENT-1347 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.5 > Environment: ARCH Linux kernel 3.8.8-1 > node.js 0.8.22 >Reporter: Adam Patacchiola >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > Attachments: Archive.zip, Screen Shot 2014-01-11 at 7.11.36 PM.png, > Screen Shot 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png, > httpClientCacheTest.tar.gz, httpClientTestServer.js, output.out > > > Compressed responses are cached twice. > Run the attached server (node.js 0.8.22) and client tests. Create an "assets" > directory under where you are running the server and add two files named 1 > and 2 ( < 100 bytes) . You will see that after the test is run the cache > dump output displays 2 sets of entries for each request, each containing the > full content length of the file. > Changing the implementation of HttpCacheStorage updateEntry to not update non > existent entries (as I believe the correct implementation should do) throws > exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2275) Implement Heuristic Expiration and Warning Code 113 for Cached Responses
Arturo Bernal created HTTPCLIENT-2275: - Summary: Implement Heuristic Expiration and Warning Code 113 for Cached Responses Key: HTTPCLIENT-2275 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2275 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal As per RFC 7234, our caching mechanism needs to implement heuristic expiration and provide appropriate warning codes when serving stale responses. The changes will ensure that we are compliant with the HTTP/1.1 cache specification and provide more informative responses to clients when serving cached data Tasks include: # Add Warning Code 113 (Heuristic expiration) when serving a cached response past its freshness lifetime. # _... to determinate._ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2273) Improve handling of "no-cache" directive with specified header fields
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2273: -- Description: Currently, the Apache HttpComponents client does not fully handle the "no-cache" directive with specified header fields as outlined in RFC 7234. This may lead to incorrect caching behavior when the "no-cache" directive is used with specific header fields. The goal of this ticket is to enhance the handling of the "no-cache" directive to ensure that specified header fields are not sent in a response to a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent the re-use of certain header fields in a response, while still allowing caching of the rest of the response. To achieve this, the following changes should be made: # Update the caching module to identify and handle "no-cache" directives with specified header fields. # Revalidate the response with the origin server when the specified header fields are present in a cached response, to ensure that they are not reused without successful revalidation. # Ensure that the rest of the response is still cacheable when the specified header fields are present, as long as the response complies with other caching requirements. was: Currently, the Apache HttpComponents client does not fully handle the "no-cache" directive with specified header fields as outlined in RFC 9111. This may lead to incorrect caching behavior when the "no-cache" directive is used with specific header fields. The goal of this ticket is to enhance the handling of the "no-cache" directive to ensure that specified header fields are either excluded from a subsequent response or revalidated with the origin server, as per RFC 9111 requirements. This allows an origin server to prevent the reuse of certain header fields in a response while still allowing caching of the rest of the response. To achieve this, the following changes should be made: # Update the caching module to identify and handle "no-cache" directives with specified header fields (qualified form). # When specified header fields are present in a cached response, either: a. Optionally attempt to revalidate the response with the origin server and update or remove the specified header fields as needed, or b. Exclude the specified header fields from the subsequent response. # Ensure that the rest of the response is still cacheable when the specified header fields are present, as long as the response complies with other caching requirements. > Improve handling of "no-cache" directive with specified header fields > - > > Key: HTTPCLIENT-2273 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2273 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > > Currently, the Apache HttpComponents client does not fully handle the > "no-cache" directive with specified header fields as outlined in RFC 7234. > This may lead to incorrect caching behavior when the "no-cache" directive is > used with specific header fields. > The goal of this ticket is to enhance the handling of the "no-cache" > directive to ensure that specified header fields are not sent in a response > to a subsequent request without successful revalidation with the origin > server. This allows an origin server to prevent the re-use of certain header > fields in a response, while still allowing caching of the rest of the > response. > To achieve this, the following changes should be made: > # Update the caching module to identify and handle "no-cache" directives > with specified header fields. > # Revalidate the response with the origin server when the specified header > fields are present in a cached response, to ensure that they are not reused > without successful revalidation. > # Ensure that the rest of the response is still cacheable when the specified > header fields are present, as long as the response complies with other > caching requirements. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2273) Improve handling of "no-cache" directive with specified header fields
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720999#comment-17720999 ] Arturo Bernal commented on HTTPCLIENT-2273: --- HI [~olegk] fair enough. > Improve handling of "no-cache" directive with specified header fields > - > > Key: HTTPCLIENT-2273 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2273 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > > Currently, the Apache HttpComponents client does not fully handle the > "no-cache" directive with specified header fields as outlined in RFC 9111. > This may lead to incorrect caching behavior when the "no-cache" directive is > used with specific header fields. > The goal of this ticket is to enhance the handling of the "no-cache" > directive to ensure that specified header fields are either excluded from a > subsequent response or revalidated with the origin server, as per RFC 9111 > requirements. This allows an origin server to prevent the reuse of certain > header fields in a response while still allowing caching of the rest of the > response. > To achieve this, the following changes should be made: > # Update the caching module to identify and handle "no-cache" directives > with specified header fields (qualified form). > # When specified header fields are present in a cached response, either: a. > Optionally attempt to revalidate the response with the origin server and > update or remove the specified header fields as needed, or b. Exclude the > specified header fields from the subsequent response. > # Ensure that the rest of the response is still cacheable when the specified > header fields are present, as long as the response complies with other > caching requirements. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2273) Improve handling of "no-cache" directive with specified header fields
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720997#comment-17720997
]
Arturo Bernal commented on HTTPCLIENT-2273:
---
Thank you [~reschke]
One doubt that I came cross its about the use of field-names as arguments in
the no-cache directive, as described in the RFC.
Is it valid to use field-names as arguments in the no-cache directive like
this: {{{}no-cache="Set-Cookie, Content-Language"{}}}? If so, can you please
provide some clarification on how this should be handled in terms of
revalidation and caching behavior?
Thank you in advance for your assistance,
> Improve handling of "no-cache" directive with specified header fields
> -
>
> Key: HTTPCLIENT-2273
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2273
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Arturo Bernal
>Priority: Major
>
> Currently, the Apache HttpComponents client does not fully handle the
> "no-cache" directive with specified header fields as outlined in RFC 9111.
> This may lead to incorrect caching behavior when the "no-cache" directive is
> used with specific header fields.
> The goal of this ticket is to enhance the handling of the "no-cache"
> directive to ensure that specified header fields are either excluded from a
> subsequent response or revalidated with the origin server, as per RFC 9111
> requirements. This allows an origin server to prevent the reuse of certain
> header fields in a response while still allowing caching of the rest of the
> response.
> To achieve this, the following changes should be made:
> # Update the caching module to identify and handle "no-cache" directives
> with specified header fields (qualified form).
> # When specified header fields are present in a cached response, either: a.
> Optionally attempt to revalidate the response with the origin server and
> update or remove the specified header fields as needed, or b. Exclude the
> specified header fields from the subsequent response.
> # Ensure that the rest of the response is still cacheable when the specified
> header fields are present, as long as the response complies with other
> caching requirements.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2273) Improve handling of "no-cache" directive with specified header fields
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2273: -- Description: Currently, the Apache HttpComponents client does not fully handle the "no-cache" directive with specified header fields as outlined in RFC 9111. This may lead to incorrect caching behavior when the "no-cache" directive is used with specific header fields. The goal of this ticket is to enhance the handling of the "no-cache" directive to ensure that specified header fields are either excluded from a subsequent response or revalidated with the origin server, as per RFC 9111 requirements. This allows an origin server to prevent the reuse of certain header fields in a response while still allowing caching of the rest of the response. To achieve this, the following changes should be made: # Update the caching module to identify and handle "no-cache" directives with specified header fields (qualified form). # When specified header fields are present in a cached response, either: a. Optionally attempt to revalidate the response with the origin server and update or remove the specified header fields as needed, or b. Exclude the specified header fields from the subsequent response. # Ensure that the rest of the response is still cacheable when the specified header fields are present, as long as the response complies with other caching requirements. was: Currently, the Apache HttpComponents client does not fully handle the "no-cache" directive with specified header fields as outlined in RFC 7234. This may lead to incorrect caching behavior when the "no-cache" directive is used with specific header fields. The goal of this ticket is to enhance the handling of the "no-cache" directive to ensure that specified header fields are not sent in a response to a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent the re-use of certain header fields in a response, while still allowing caching of the rest of the response. To achieve this, the following changes should be made: # Update the caching module to identify and handle "no-cache" directives with specified header fields. # Revalidate the response with the origin server when the specified header fields are present in a cached response, to ensure that they are not reused without successful revalidation. # Ensure that the rest of the response is still cacheable when the specified header fields are present, as long as the response complies with other caching requirements. > Improve handling of "no-cache" directive with specified header fields > - > > Key: HTTPCLIENT-2273 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2273 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Major > > Currently, the Apache HttpComponents client does not fully handle the > "no-cache" directive with specified header fields as outlined in RFC 9111. > This may lead to incorrect caching behavior when the "no-cache" directive is > used with specific header fields. > The goal of this ticket is to enhance the handling of the "no-cache" > directive to ensure that specified header fields are either excluded from a > subsequent response or revalidated with the origin server, as per RFC 9111 > requirements. This allows an origin server to prevent the reuse of certain > header fields in a response while still allowing caching of the rest of the > response. > To achieve this, the following changes should be made: > # Update the caching module to identify and handle "no-cache" directives > with specified header fields (qualified form). > # When specified header fields are present in a cached response, either: a. > Optionally attempt to revalidate the response with the origin server and > update or remove the specified header fields as needed, or b. Exclude the > specified header fields from the subsequent response. > # Ensure that the rest of the response is still cacheable when the specified > header fields are present, as long as the response complies with other > caching requirements. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2273) Improve handling of "no-cache" directive with specified header fields
Arturo Bernal created HTTPCLIENT-2273: - Summary: Improve handling of "no-cache" directive with specified header fields Key: HTTPCLIENT-2273 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2273 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal Currently, the Apache HttpComponents client does not fully handle the "no-cache" directive with specified header fields as outlined in RFC 7234. This may lead to incorrect caching behavior when the "no-cache" directive is used with specific header fields. The goal of this ticket is to enhance the handling of the "no-cache" directive to ensure that specified header fields are not sent in a response to a subsequent request without successful revalidation with the origin server. This allows an origin server to prevent the re-use of certain header fields in a response, while still allowing caching of the rest of the response. To achieve this, the following changes should be made: # Update the caching module to identify and handle "no-cache" directives with specified header fields. # Revalidate the response with the origin server when the specified header fields are present in a cached response, to ensure that they are not reused without successful revalidation. # Ensure that the rest of the response is still cacheable when the specified header fields are present, as long as the response complies with other caching requirements. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1713) Certain responses from a POST should be cacheable
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720288#comment-17720288
]
Arturo Bernal commented on HTTPCLIENT-1713:
---
[~olegk]
This can be close
[https://github.com/apache/httpcomponents-client/commit/f7ff1088cf4610613ec4b867ebc32021238f898a]
> Certain responses from a POST should be cacheable
> -
>
> Key: HTTPCLIENT-1713
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1713
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpCache
>Reporter: Brian Chang
>Priority: Minor
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
>
> In {{ResponseCachingPolicy}}, only {{GET}} and {{HEAD}} methods are
> cacheable. However, according to RFC 2616
> [https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5] a {{POST}}
> may be cacheable.
> The exact circumstances are better detailed here:
> [http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-20#section-2.3.4]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1725) Heuristic caching does not work for URIs with a query string
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720283#comment-17720283
]
Arturo Bernal commented on HTTPCLIENT-1725:
---
@[~olegk]
can be close [https://github.com/apache/httpcomponents-client/pull/417]
> Heuristic caching does not work for URIs with a query string
>
>
> Key: HTTPCLIENT-1725
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1725
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpCache
>Affects Versions: 4.5.1
>Reporter: Charlie Halford
>Priority: Major
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
> Attachments: HTTPCLIENT-1725_add_query_string_caching.diff
>
>
> When enabling heuristic caching and setting a default lifetime, the responses
> from the server I am requesting from are not being stored in the cache.
> In org.apache.http.impl.client.cache.ResponseCachingPolicy, line 250
> determines if the URI contains a query string:
> {code}
> request.getRequestLine().getUri().contains("?")
> {code}
> A few lines below, it then checks to see if the response contains cache
> headers:
> {code}
> else if (!isExplicitlyCacheable(response))
> {code}
> As I am attempting to cache a response that I know does not contain cache
> headers, the response should succeed in being cached. However, it fails the
> isExplictlyCachable() check, and thus returns false to the overall
> isResponseCachable() method. The isExplicitlyCachable() method is checked
> later on in the stack, so perhaps it can be safely removed here?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2272) Add Exponential Backoff Strategy and Linear Strategy to AIMDBackoffManager
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal updated HTTPCLIENT-2272:
--
Description:
Exponential Backoff Strategy:
As part of our ongoing efforts to improve the reliability and performance of
our system, we want to add support for exponential backoff in the
{{AIMDBackoffManager}} class.
Exponential backoff is a common strategy used in networking to reduce the
number of retries and failures during high load or congested periods. The basic
idea is to increase the delay between retries exponentially after each attempt,
instead of using a fixed delay as in linear backoff. This helps to reduce the
load on the server and improve overall system stability.
The implementation of the exponential backoff strategy will be similar to the
existing linear backoff strategy, but with some modifications to the delay
calculation. The delay between retries will be calculated using an exponential
function with a configurable base value and multiplier. The base value will be
the initial delay, and the multiplier will be the factor by which
the delay increases after each attempt.
Linear Backoff Strategies:
Currently, our AIMDBackoffManager only supports the Additive Increase
Multiplicative Decrease (AIMD) algorithm for managing connection pool sizes.
However, we would like to introduce additional Linear Backoff Strategies to
improve our connection management and provide better resilience to network
failures.
The proposed Linear Backoff Strategies will allow us to gradually increase or
decrease the connection pool size by a fixed amount in response to failed
attempts or successful probes. This approach is more predictable than AIMD and
can be more appropriate for certain network topologies or failure modes.
was:
As part of our ongoing efforts to improve the reliability and performance of
our system, we want to add support for exponential backoff in the
{{AIMDBackoffManager}} class.
Exponential backoff is a common strategy used in networking to reduce the
number of retries and failures during high load or congested periods. The basic
idea is to increase the delay between retries exponentially after each attempt,
instead of using a fixed delay as in linear backoff. This helps to reduce the
load on the server and improve overall system stability.
The implementation of the exponential backoff strategy will be similar to the
existing linear backoff strategy, but with some modifications to the delay
calculation. The delay between retries will be calculated using an exponential
function with a configurable base value and multiplier. The base value will be
the initial delay, and the multiplier will be the factor by which the delay
increases after each attempt.
> Add Exponential Backoff Strategy and Linear Strategy to AIMDBackoffManager
> ---
>
> Key: HTTPCLIENT-2272
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2272
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Arturo Bernal
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Exponential Backoff Strategy:
>
> As part of our ongoing efforts to improve the reliability and performance of
> our system, we want to add support for exponential backoff in the
> {{AIMDBackoffManager}} class.
> Exponential backoff is a common strategy used in networking to reduce the
> number of retries and failures during high load or congested periods. The
> basic idea is to increase the delay between retries exponentially after each
> attempt, instead of using a fixed delay as in linear backoff. This helps to
> reduce the load on the server and improve overall system stability.
> The implementation of the exponential backoff strategy will be similar to the
> existing linear backoff strategy, but with some modifications to the delay
> calculation. The delay between retries will be calculated using an
> exponential function with a configurable base value and multiplier. The base
> value will be the initial delay, and the multiplier will be the factor by
> which
> the delay increases after each attempt.
>
> Linear Backoff Strategies:
>
> Currently, our AIMDBackoffManager only supports the Additive Increase
> Multiplicative Decrease (AIMD) algorithm for managing connection pool sizes.
> However, we would like to introduce additional Linear Backoff Strategies to
> improve our connection management and provide better resilience to network
> failures.
> The proposed Linear Backoff Strategies will allow us to gradually increase or
> decrease the connection pool size by a fixed amount in response to failed
> attempts or successful probes. This approach is more predictable than AIMD
> and can be
[jira] [Updated] (HTTPCLIENT-2272) Add Exponential Backoff Strategy and Linear Strategy to AIMDBackoffManager
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arturo Bernal updated HTTPCLIENT-2272:
--
Summary: Add Exponential Backoff Strategy and Linear Strategy to
AIMDBackoffManager (was: Add Exponential Backoff Strategy to
AIMDBackoffManager)
> Add Exponential Backoff Strategy and Linear Strategy to AIMDBackoffManager
> ---
>
> Key: HTTPCLIENT-2272
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2272
> Project: HttpComponents HttpClient
> Issue Type: Improvement
>Reporter: Arturo Bernal
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> As part of our ongoing efforts to improve the reliability and performance of
> our system, we want to add support for exponential backoff in the
> {{AIMDBackoffManager}} class.
> Exponential backoff is a common strategy used in networking to reduce the
> number of retries and failures during high load or congested periods. The
> basic idea is to increase the delay between retries exponentially after each
> attempt, instead of using a fixed delay as in linear backoff. This helps to
> reduce the load on the server and improve overall system stability.
> The implementation of the exponential backoff strategy will be similar to the
> existing linear backoff strategy, but with some modifications to the delay
> calculation. The delay between retries will be calculated using an
> exponential function with a configurable base value and multiplier. The base
> value will be the initial delay, and the multiplier will be the factor by
> which the delay increases after each attempt.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Closed] (HTTPCLIENT-2270) Linear Backoff Strategies for AIMDBackoffManager
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal closed HTTPCLIENT-2270. - Resolution: Fixed Cose and reuse one ticket https://issues.apache.org/jira/browse/HTTPCLIENT-2272 > Linear Backoff Strategies for AIMDBackoffManager > > > Key: HTTPCLIENT-2270 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2270 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Currently, our AIMDBackoffManager only supports the Additive Increase > Multiplicative Decrease (AIMD) algorithm for managing connection pool sizes. > However, we would like to introduce additional Linear Backoff Strategies to > improve our connection management and provide better resilience to network > failures. > The proposed Linear Backoff Strategies will allow us to gradually increase or > decrease the connection pool size by a fixed amount in response to failed > attempts or successful probes. This approach is more predictable than AIMD > and can be more appropriate for certain network topologies or failure modes. > We plan to implement two new Linear Backoff Strategies: Linear Incremental > and Linear Exponential. Linear Incremental will adjust the connection pool > size by a fixed increment, while Linear Exponential will adjust the size by a > percentage of the current size. Both strategies will have configurable > parameters to customize the increment or percentage, the cool-down period, > and the maximum and minimum size of the pool -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCORE-478) Add "Forwarded" HttpRequestInterceptor
[ https://issues.apache.org/jira/browse/HTTPCORE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718944#comment-17718944 ] Arturo Bernal edited comment on HTTPCORE-478 at 5/3/23 2:05 PM: [~olegk] This can be closed. The implementation has been completed in the 5.3.X branch. [https://github.com/apache/httpcomponents-core/pull/390] was (Author: arturobernalg): [~olegk] This can be close. [https://github.com/apache/httpcomponents-core/pull/390] > Add "Forwarded" HttpRequestInterceptor > -- > > Key: HTTPCORE-478 > URL: https://issues.apache.org/jira/browse/HTTPCORE-478 > Project: HttpComponents HttpCore > Issue Type: New Feature >Reporter: Gary D. Gregory >Priority: Major > > Add a "Forwarded" HttpRequestInterceptor to add the "Forwarded" HTTP header. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-478) Add "Forwarded" HttpRequestInterceptor
[ https://issues.apache.org/jira/browse/HTTPCORE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17718944#comment-17718944 ] Arturo Bernal commented on HTTPCORE-478: [~olegk] This can be close. [https://github.com/apache/httpcomponents-core/pull/390] > Add "Forwarded" HttpRequestInterceptor > -- > > Key: HTTPCORE-478 > URL: https://issues.apache.org/jira/browse/HTTPCORE-478 > Project: HttpComponents HttpCore > Issue Type: New Feature >Reporter: Gary D. Gregory >Priority: Major > > Add a "Forwarded" HttpRequestInterceptor to add the "Forwarded" HTTP header. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2270) Linear Backoff Strategies for AIMDBackoffManager
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2270: -- Description: Currently, our AIMDBackoffManager only supports the Additive Increase Multiplicative Decrease (AIMD) algorithm for managing connection pool sizes. However, we would like to introduce additional Linear Backoff Strategies to improve our connection management and provide better resilience to network failures. The proposed Linear Backoff Strategies will allow us to gradually increase or decrease the connection pool size by a fixed amount in response to failed attempts or successful probes. This approach is more predictable than AIMD and can be more appropriate for certain network topologies or failure modes. We plan to implement two new Linear Backoff Strategies: Linear Incremental and Linear Exponential. Linear Incremental will adjust the connection pool size by a fixed increment, while Linear Exponential will adjust the size by a percentage of the current size. Both strategies will have configurable parameters to customize the increment or percentage, the cool-down period, and the maximum and minimum size of the pool was: In order to offer more flexibility and options for backoff strategies in the AIMDBackoffManager, we'd like to introduceLinear Backoff strategie. These new strategie can be useful in various networking scenarios and cater to different use cases. Tasks: # Design and implement the Exponential Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Design and implement the Linear Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Write unit tests to validate the functionality of the new Exponential and Linear Backoff strategies. # Update the documentation and JavaDoc to include the new backoff strategies and provide usage examples. # Ensure that the new strategies are compatible with the existing AIMDBackoffManager implementation and do not break any existing functionality > Linear Backoff Strategies for AIMDBackoffManager > > > Key: HTTPCLIENT-2270 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2270 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > > Currently, our AIMDBackoffManager only supports the Additive Increase > Multiplicative Decrease (AIMD) algorithm for managing connection pool sizes. > However, we would like to introduce additional Linear Backoff Strategies to > improve our connection management and provide better resilience to network > failures. > The proposed Linear Backoff Strategies will allow us to gradually increase or > decrease the connection pool size by a fixed amount in response to failed > attempts or successful probes. This approach is more predictable than AIMD > and can be more appropriate for certain network topologies or failure modes. > We plan to implement two new Linear Backoff Strategies: Linear Incremental > and Linear Exponential. Linear Incremental will adjust the connection pool > size by a fixed increment, while Linear Exponential will adjust the size by a > percentage of the current size. Both strategies will have configurable > parameters to customize the increment or percentage, the cool-down period, > and the maximum and minimum size of the pool -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2272) Add Exponential Backoff Strategy to AIMDBackoffManager
Arturo Bernal created HTTPCLIENT-2272:
-
Summary: Add Exponential Backoff Strategy to AIMDBackoffManager
Key: HTTPCLIENT-2272
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2272
Project: HttpComponents HttpClient
Issue Type: Improvement
Reporter: Arturo Bernal
As part of our ongoing efforts to improve the reliability and performance of
our system, we want to add support for exponential backoff in the
{{AIMDBackoffManager}} class.
Exponential backoff is a common strategy used in networking to reduce the
number of retries and failures during high load or congested periods. The basic
idea is to increase the delay between retries exponentially after each attempt,
instead of using a fixed delay as in linear backoff. This helps to reduce the
load on the server and improve overall system stability.
The implementation of the exponential backoff strategy will be similar to the
existing linear backoff strategy, but with some modifications to the delay
calculation. The delay between retries will be calculated using an exponential
function with a configurable base value and multiplier. The base value will be
the initial delay, and the multiplier will be the factor by which the delay
increases after each attempt.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2270) Linear Backoff Strategies for AIMDBackoffManager
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2270: -- Summary: Linear Backoff Strategies for AIMDBackoffManager (was: Implement Exponential and Linear Backoff Strategies for AIMDBackoffManager) > Linear Backoff Strategies for AIMDBackoffManager > > > Key: HTTPCLIENT-2270 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2270 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > > In order to offer more flexibility and options for backoff strategies in the > AIMDBackoffManager, we'd like to introduce Exponential and Linear Backoff > strategies. These new strategies can be useful in various networking > scenarios and cater to different use cases. > Tasks: > # Design and implement the Exponential Backoff strategy based on the > customizable backoff strategy interface (after implementing customizable > strategies). > # Design and implement the Linear Backoff strategy based on the customizable > backoff strategy interface (after implementing customizable strategies). > # Write unit tests to validate the functionality of the new Exponential and > Linear Backoff strategies. > # Update the documentation and JavaDoc to include the new backoff strategies > and provide usage examples. > # Ensure that the new strategies are compatible with the existing > AIMDBackoffManager implementation and do not break any existing functionality -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Updated] (HTTPCLIENT-2270) Linear Backoff Strategies for AIMDBackoffManager
[ https://issues.apache.org/jira/browse/HTTPCLIENT-2270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arturo Bernal updated HTTPCLIENT-2270: -- Description: In order to offer more flexibility and options for backoff strategies in the AIMDBackoffManager, we'd like to introduceLinear Backoff strategie. These new strategie can be useful in various networking scenarios and cater to different use cases. Tasks: # Design and implement the Exponential Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Design and implement the Linear Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Write unit tests to validate the functionality of the new Exponential and Linear Backoff strategies. # Update the documentation and JavaDoc to include the new backoff strategies and provide usage examples. # Ensure that the new strategies are compatible with the existing AIMDBackoffManager implementation and do not break any existing functionality was: In order to offer more flexibility and options for backoff strategies in the AIMDBackoffManager, we'd like to introduce Exponential and Linear Backoff strategies. These new strategies can be useful in various networking scenarios and cater to different use cases. Tasks: # Design and implement the Exponential Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Design and implement the Linear Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Write unit tests to validate the functionality of the new Exponential and Linear Backoff strategies. # Update the documentation and JavaDoc to include the new backoff strategies and provide usage examples. # Ensure that the new strategies are compatible with the existing AIMDBackoffManager implementation and do not break any existing functionality > Linear Backoff Strategies for AIMDBackoffManager > > > Key: HTTPCLIENT-2270 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2270 > Project: HttpComponents HttpClient > Issue Type: Improvement >Reporter: Arturo Bernal >Priority: Minor > > In order to offer more flexibility and options for backoff strategies in the > AIMDBackoffManager, we'd like to introduceLinear Backoff strategie. These new > strategie can be useful in various networking scenarios and cater to > different use cases. > Tasks: > # Design and implement the Exponential Backoff strategy based on the > customizable backoff strategy interface (after implementing customizable > strategies). > # Design and implement the Linear Backoff strategy based on the customizable > backoff strategy interface (after implementing customizable strategies). > # Write unit tests to validate the functionality of the new Exponential and > Linear Backoff strategies. > # Update the documentation and JavaDoc to include the new backoff strategies > and provide usage examples. > # Ensure that the new strategies are compatible with the existing > AIMDBackoffManager implementation and do not break any existing functionality -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1347) gzip responses doubly cached
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717653#comment-17717653 ] Arturo Bernal commented on HTTPCLIENT-1347: --- This still happening with the 5.3 version? [~madman6000] > gzip responses doubly cached > > > Key: HTTPCLIENT-1347 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1347 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.5 > Environment: ARCH Linux kernel 3.8.8-1 > node.js 0.8.22 >Reporter: Adam Patacchiola >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > Attachments: Screen Shot 2014-01-11 at 7.11.36 PM.png, Screen Shot > 2014-01-13 at 3.56.19 PM.png, Showing_entry_pointer.png, > httpClientCacheTest.tar.gz, httpClientTestServer.js > > > Compressed responses are cached twice. > Run the attached server (node.js 0.8.22) and client tests. Create an "assets" > directory under where you are running the server and add two files named 1 > and 2 ( < 100 bytes) . You will see that after the test is run the cache > dump output displays 2 sets of entries for each request, each containing the > full content length of the file. > Changing the implementation of HttpCacheStorage updateEntry to not update non > existent entries (as I believe the correct implementation should do) throws > exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1580) Deprecate NTCredentials constructor with workstation parameter
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17717624#comment-17717624 ] Arturo Bernal commented on HTTPCLIENT-1580: --- TY > Deprecate NTCredentials constructor with workstation parameter > -- > > Key: HTTPCLIENT-1580 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1580 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Affects Versions: 4.4 Beta1 >Reporter: Michael Osipov >Assignee: Michael Osipov >Priority: Major > Labels: volunteers-wanted > Fix For: 5.3-alpha1 > > > The underlying NTLM implemntation shall determine the localname of the > current workstation at runtime/request time. There is no need to supply this > information. Other popular implementation do not provide this attribute > either, e.g., curl. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1874) Exponential back-off retry mechanism
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716919#comment-17716919 ] Arturo Bernal commented on HTTPCLIENT-1874: --- can we close this ticket [~olegk] ? > Exponential back-off retry mechanism > > > Key: HTTPCLIENT-1874 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1874 > Project: HttpComponents HttpClient > Issue Type: New Feature >Reporter: Ioannis Sermetziadis >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > > I believe it would be useful to have an Exponential Back-off algorithm > implementation, as an HttpRequestRetryHandler or under a new interface, named > HttpRequestRetryPolicy. > The implementation could provide both synchronous (similar to the existing > retry mechanism) and asynchronous retries, using a provided > ExecutorService/ScheduledExecutorService. > Any feedback would be much appreciated. If you agree with the value of such a > feature, I could proceed to the implementation. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1920) Cached HEAD requests without response body
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716918#comment-17716918
]
Arturo Bernal commented on HTTPCLIENT-1920:
---
[https://github.com/apache/httpcomponents-client/pull/433]
> Cached HEAD requests without response body
> --
>
> Key: HTTPCLIENT-1920
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1920
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.4.1, 4.5
>Reporter: Tobias Gesellchen
>Priority: Major
> Labels: pull-request-available
>
> Since httpclient-cache 4.4 {{HEAD}} requests can be cached with a {{null}}
> response body. Subsequent {{GET}} requests to the same target then use the
> headers "If-None-Match" and "If-Modified-Since" to leverage caching via HTTP
> protocol. In that case a server can respond with status code 304 and an empty
> response body, which leads to an invalid cache entry.
> The GitHub PR #101 proposes a fix, which I have tested locally. The PR is
> based on httpclient-cache 4.5, but should easily be backported to 4.4. I
> haven't looked into the upcoming 5.x sources, yet.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1953) HTTP Response is not cached with Cache-Control: no-cache header
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716896#comment-17716896 ] Arturo Bernal commented on HTTPCLIENT-1953: --- [~olegk] I guess this too. > HTTP Response is not cached with Cache-Control: no-cache header > --- > > Key: HTTPCLIENT-1953 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1953 > Project: HttpComponents HttpClient > Issue Type: Bug > Components: HttpCache >Affects Versions: 4.5.6 >Reporter: Marco Signorini >Priority: Major > Labels: stuck, volunteers-wanted > Fix For: Stuck > > Attachments: WebClientTest.java > > > When the HTTP response contains the Cache-Control: no-cache header, the > response should be cached and then validated against the origin server before > returning it. > Now the HTTP response is never cached. > [https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching] > [https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-1756) Response Body on response codes over 300 should not be ignored
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716882#comment-17716882
]
Arturo Bernal edited comment on HTTPCLIENT-1756 at 4/26/23 7:41 PM:
[~olegk]
can be close.
[https://github.com/apache/httpcomponents-client/pull/423]
was (Author: arturobernalg):
[~olegk]
can be close.
> Response Body on response codes over 300 should not be ignored
> --
>
> Key: HTTPCLIENT-1756
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1756
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.2
>Reporter: Michael Peter Gower
>Priority: Minor
> Fix For: Future
>
>
> Hello. This is my first time posting an issue to this project. I've done my
> best to be helpful.
> Many modern RESTful APIs (e.g. Amazon Web Services) will return a generic
> status description (e.g. "Bad Request") and details in the body of the
> response on an error (e.g. code 400).
> However the org.apache.http.client.fluent.Response.returnContent() ignores
> the body (content) on a return code of over or equal to 300, causing the
> debugging information to be lost. This leads to generic exception messages.
> A possible solution would be to include the body in the exception string, as
> demonstrated in the below ResponseHandler.
> {code:java}
> protected static class FaultTolerantContentResponseHandler
> extends ContentResponseHandler {
> @Override
> public Content handleResponse(final HttpResponse response)
> throws HttpResponseException, IOException {
> final StatusLine statusLine = response.getStatusLine();
> final HttpEntity entity = response.getEntity();
> Content content = entity == null ? null : handleEntity(entity);
> if (statusLine.getStatusCode() >= 300) {
> String message = statusLine.getReasonPhrase();
> if (content != null) {
> message += ", body was " + content.asString();
> }
> throw new HttpResponseException(statusLine.getStatusCode(),
> message);
> }
> return content;
> }
> }
> {code}
> Alternatives would be to create a new subclass of HttpResponseException that
> would have an additional getErrorContent() method to preserve backwards
> compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-1756) Response Body on response codes over 300 should not be ignored
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716882#comment-17716882
]
Arturo Bernal edited comment on HTTPCLIENT-1756 at 4/26/23 7:39 PM:
[~olegk]
can be close.
was (Author: arturobernalg):
[~olegk]
can be close.
[https://github.com/apache/httpcomponents-client/pull/430]
> Response Body on response codes over 300 should not be ignored
> --
>
> Key: HTTPCLIENT-1756
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1756
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.2
>Reporter: Michael Peter Gower
>Priority: Minor
> Fix For: Future
>
>
> Hello. This is my first time posting an issue to this project. I've done my
> best to be helpful.
> Many modern RESTful APIs (e.g. Amazon Web Services) will return a generic
> status description (e.g. "Bad Request") and details in the body of the
> response on an error (e.g. code 400).
> However the org.apache.http.client.fluent.Response.returnContent() ignores
> the body (content) on a return code of over or equal to 300, causing the
> debugging information to be lost. This leads to generic exception messages.
> A possible solution would be to include the body in the exception string, as
> demonstrated in the below ResponseHandler.
> {code:java}
> protected static class FaultTolerantContentResponseHandler
> extends ContentResponseHandler {
> @Override
> public Content handleResponse(final HttpResponse response)
> throws HttpResponseException, IOException {
> final StatusLine statusLine = response.getStatusLine();
> final HttpEntity entity = response.getEntity();
> Content content = entity == null ? null : handleEntity(entity);
> if (statusLine.getStatusCode() >= 300) {
> String message = statusLine.getReasonPhrase();
> if (content != null) {
> message += ", body was " + content.asString();
> }
> throw new HttpResponseException(statusLine.getStatusCode(),
> message);
> }
> return content;
> }
> }
> {code}
> Alternatives would be to create a new subclass of HttpResponseException that
> would have an additional getErrorContent() method to preserve backwards
> compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1176) Support proxy failover per ProxySelector API
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716887#comment-17716887 ] Arturo Bernal commented on HTTPCLIENT-1176: --- Can be close [https://github.com/apache/httpcomponents-client/pull/430] > Support proxy failover per ProxySelector API > > > Key: HTTPCLIENT-1176 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1176 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpClient (classic) >Affects Versions: 4.2 Beta1 > Environment: All supported platforms/environments which use proxy > failover >Reporter: Jan-Willem Maarse >Priority: Minor > Fix For: Stuck > > Attachments: ProxyFailoverFix.patch, ProxyFailoverTest.patch > > > The ProxySelectorRoutePlanner selects a single proxy from the list returned > by the java.net.ProxySelector API. If the client can't open a socket to the > proxy, other proxies in the list are ignored and the request fails. It would > be good if the Apache HttpClient could try other proxies in the list before > failing on socket errors. HttpURLConnection provides similar behavior. This > functionality is useful in network configurations where proxies can go up and > down for maintenance, for example. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1469) Option to Include Last-Modified in 304 Not Modified Response
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716886#comment-17716886 ] Arturo Bernal commented on HTTPCLIENT-1469: --- [~olegk] can be close [https://github.com/apache/httpcomponents-client/pull/435] > Option to Include Last-Modified in 304 Not Modified Response > > > Key: HTTPCLIENT-1469 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1469 > Project: HttpComponents HttpClient > Issue Type: New Feature > Components: HttpCache >Affects Versions: 4.3.2 >Reporter: James Leigh >Priority: Minor > Labels: Stuck, volunteers-wanted > Fix For: Stuck > > Attachments: HTTPCLIENT-1469-httpasyncclient.patch, > HTTPCLIENT-1469.patch > > > The draft of the revised HTTP/1.1 (should be RFC'd shortly) says other > response header can be included in 304 responses if they exists for the > purpose of guiding cache updates. > http://tools.ietf.org/html/draft-ietf-httpbis-p4-conditional-26#section-4.1 >The server generating a 304 response MUST generate any of the >following header fields that would have been sent in a 200 (OK) >response to the same request: Cache-Control, Content-Location, Date, >ETag, Expires, and Vary. >Since the goal of a 304 response is to minimize information transfer >when the recipient already has one or more cached representations, a >sender SHOULD NOT generate representation metadata other than the >above listed fields unless said metadata exists for the purpose of >guiding cache updates (e.g., Last-Modified might be useful if the >response does not have an ETag field). > This issue is to make it possible to include Last-Modified in a cached 304 > response. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1756) Response Body on response codes over 300 should not be ignored
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716882#comment-17716882
]
Arturo Bernal commented on HTTPCLIENT-1756:
---
[~olegk]
can be close.
[https://github.com/apache/httpcomponents-client/pull/430]
> Response Body on response codes over 300 should not be ignored
> --
>
> Key: HTTPCLIENT-1756
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1756
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 4.5.2
>Reporter: Michael Peter Gower
>Priority: Minor
> Fix For: Future
>
>
> Hello. This is my first time posting an issue to this project. I've done my
> best to be helpful.
> Many modern RESTful APIs (e.g. Amazon Web Services) will return a generic
> status description (e.g. "Bad Request") and details in the body of the
> response on an error (e.g. code 400).
> However the org.apache.http.client.fluent.Response.returnContent() ignores
> the body (content) on a return code of over or equal to 300, causing the
> debugging information to be lost. This leads to generic exception messages.
> A possible solution would be to include the body in the exception string, as
> demonstrated in the below ResponseHandler.
> {code:java}
> protected static class FaultTolerantContentResponseHandler
> extends ContentResponseHandler {
> @Override
> public Content handleResponse(final HttpResponse response)
> throws HttpResponseException, IOException {
> final StatusLine statusLine = response.getStatusLine();
> final HttpEntity entity = response.getEntity();
> Content content = entity == null ? null : handleEntity(entity);
> if (statusLine.getStatusCode() >= 300) {
> String message = statusLine.getReasonPhrase();
> if (content != null) {
> message += ", body was " + content.asString();
> }
> throw new HttpResponseException(statusLine.getStatusCode(),
> message);
> }
> return content;
> }
> }
> {code}
> Alternatives would be to create a new subclass of HttpResponseException that
> would have an additional getErrorContent() method to preserve backwards
> compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2185) Add support for IDNA 2008 (RFC 5891)
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716426#comment-17716426
]
Arturo Bernal commented on HTTPCLIENT-2185:
---
[~olegk] this can be closed as well.
> Add support for IDNA 2008 (RFC 5891)
>
>
> Key: HTTPCLIENT-2185
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2185
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: Android Port, HttpClient (async), HttpClient (classic),
> HttpClient (Windows)
>Affects Versions: 5.0
> Environment: Android SDK 30
>Reporter: Julien Bernard
>Priority: Minor
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> The tests have been conducted with Kotlin on Android SDK 30 with Apache
> HttpClient Android extensions which ships with HttpClient version 5.0.1 but
> any HttpClient component on any platform may be impacted.
> Before making an HTTP query the client should convert the URL domain to
> A-label as described in [RFC
> 5891|https://datatracker.ietf.org/doc/html/rfc5891].
> Here is an example with the trustworthy ICU4J library:
>
> {code:java}
> int flags = IDNA.CHECK_BIDI
>| IDNA.CHECK_CONTEXTJ
>| IDNA.CHECK_CONTEXTO
>| IDNA.NONTRANSITIONAL_TO_ASCII
>| IDNA.USE_STD3_RULES;
> IDNA idna = IDNA.getUTS46Instance(flags);
> IDNA.Info info = new IDNA.Info();
> String domainAscii = idna.nameToASCII(domain, new StringBuilder(), info);
> // check error with info.hasErrors() {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1795) Http Client does not support RFC-compliant quoted IPv6 Link-Local host literals
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716250#comment-17716250
]
Arturo Bernal commented on HTTPCLIENT-1795:
---
[~olegk]
This can be close it
[ae46fd3|https://github.com/apache/httpcomponents-client/commit/ae46fd3561899b89fee0559e32c30881e04fc71f]
> Http Client does not support RFC-compliant quoted IPv6 Link-Local host
> literals
> ---
>
> Key: HTTPCLIENT-1795
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1795
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (async)
>Affects Versions: 4.5.2
>Reporter: Andreas Wundsam
>Priority: Major
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
>
> [RFC 6874|https://tools.ietf.org/html/rfc6874] states that, when constructing
> a URL with an LLv6 literal, the {{%}} sign that prefixes the ZoneID must be
> quoted as {{%25}}. E.g., the LLv6 address
> {{fe80::221:b7ff:fe8a:57d5%en4}} should be specified as
> {{scheme://[fe80::221:b7ff:fe8a:57d5%25en4]/...}} in a URL.
> httpclient does not seem to support quoted host literals:
> Example:
> {code}
> Request.Get("http://[fe80::221:b7ff:fe8a:57d5%25en4]/;)
> .connectTimeout(1000)
> .socketTimeout(1000)
> .execute().returnContent().asString();
> {code}
> results in
> {noformat}
> java.net.UnknownHostException: no such interface 25en4
> at java.net.Inet6Address.initstr(Inet6Address.java:487)
> at java.net.Inet6Address.(Inet6Address.java:408)
> at java.net.InetAddress.getAllByName(InetAddress.java:1181)
> at java.net.InetAddress.getAllByName(InetAddress.java:1126)
> at
> org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
> at
> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:111)
> at
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
> at
> org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at
> org.apache.http.client.fluent.Request.internalExecute(Request.java:173)
> at org.apache.http.client.fluent.Request.execute(Request.java:177)
> at Test.name(Test.java:12)
> [...]
> {noformat}
> It appears that httpclient directly passes the host literal
> {{[fe80::221:b7ff:fe8a:57d5%25en4]}} to the Java standard method
> {{InetAddress.getAllByName(host)}}, which does not support the URL quoting
> that RFC 6874 prescribes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1570) GGSSchemeBase blindly assumes that SPNEGO is always used
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715984#comment-17715984
]
Arturo Bernal commented on HTTPCLIENT-1570:
---
[~michael-o]
this ticket its even valid?
> GGSSchemeBase blindly assumes that SPNEGO is always used
>
>
> Key: HTTPCLIENT-1570
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1570
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (classic)
>Affects Versions: 4.4 Beta1
>Reporter: Michael Osipov
>Priority: Major
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
>
> Lines 248 to 249 say:
> {code}
> buffer.append(": Negotiate ");
> buffer.append(tokenstr);
> {code}
> This is incorrect. Since this is an abstract class the actual scheme must be
> passed by the extending class through the super constructor. Same applies for
> the {{getSchemeName}} method. Correct behavior is implemented in the
> {{httpclient-win}} module and can be used as a template.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2270) Implement Exponential and Linear Backoff Strategies for AIMDBackoffManager
Arturo Bernal created HTTPCLIENT-2270: - Summary: Implement Exponential and Linear Backoff Strategies for AIMDBackoffManager Key: HTTPCLIENT-2270 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2270 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal In order to offer more flexibility and options for backoff strategies in the AIMDBackoffManager, we'd like to introduce Exponential and Linear Backoff strategies. These new strategies can be useful in various networking scenarios and cater to different use cases. Tasks: # Design and implement the Exponential Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Design and implement the Linear Backoff strategy based on the customizable backoff strategy interface (after implementing customizable strategies). # Write unit tests to validate the functionality of the new Exponential and Linear Backoff strategies. # Update the documentation and JavaDoc to include the new backoff strategies and provide usage examples. # Ensure that the new strategies are compatible with the existing AIMDBackoffManager implementation and do not break any existing functionality -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Created] (HTTPCLIENT-2269) Implement Customizable Backoff and Probe Strategies for AIMDBackoffManager
Arturo Bernal created HTTPCLIENT-2269: - Summary: Implement Customizable Backoff and Probe Strategies for AIMDBackoffManager Key: HTTPCLIENT-2269 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2269 Project: HttpComponents HttpClient Issue Type: Improvement Reporter: Arturo Bernal As part of enhancing the AIMDBackoffManager, we'd like to introduce customizable backoff and probe strategies. This will enable users to provide their own custom strategies if needed, catering to different use cases and requirements. Tasks: # Design an interface for the backoff and probe strategies. # Refactor the existing AIMDBackoffManager implementation to use the new strategy interfaces. # Create default backoff and probe strategy implementations based on the current implementation. # Update AIMDBackoffManager to accept custom strategy implementations through its constructor or setter methods. # Write unit tests to validate the custom strategy implementations. # Update the documentation and JavaDoc to reflect the new changes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-2064) MultipartEntityBuilder should support preamble and epilog
[
https://issues.apache.org/jira/browse/HTTPCLIENT-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707287#comment-17707287
]
Arturo Bernal commented on HTTPCLIENT-2064:
---
i think we can close this ticket [~michael-o] [~olegk]
> MultipartEntityBuilder should support preamble and epilog
> -
>
> Key: HTTPCLIENT-2064
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2064
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (classic)
>Affects Versions: 5.0
>Reporter: Michael Osipov
>Priority: Major
>
> According to RFC 2046, section 5.1.1 one can augment the multipart message
> with a preamble and an epilog. Both may consist of:
>
> {noformat}
> multipart-body := [preamble CRLF]
>dash-boundary transport-padding CRLF
>body-part *encapsulation
>close-delimiter transport-padding
>[CRLF epilogue]
> preamble := discard-text
> epilogue := discard-text
> discard-text := *(*text CRLF) *text
> ; May be ignored or discarded.
> {noformat}
> {{text}} is defined in RFC 822, section 3.3 as:
> {noformat}
> text = atoms, specials,
> CR & bare LF, but NOT ; comments and
> including CRLF> ; quoted-strings are
> ; NOT recognized.
> {noformat}
> {{CHAR}} is 0 to 127.
> This could be added as
> {code}
> MultipartEntityBuilder#addPreamble()
> MultipartEntityBuilder#addEpilog()
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Comment Edited] (HTTPCLIENT-1368) stale-if-error request cache directive should also apply to non-revalidatable cache entries
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706602#comment-17706602 ] Arturo Bernal edited comment on HTTPCLIENT-1368 at 3/29/23 9:10 PM: Hi [~olegk] Thank you for taking the time to review my changes and providing feedback. You were right in pointing out that my initial implementation was not correct and could potentially lead to issues with the async protocol handling pipeline. After some further research , i made the necessary changes to ensure that stale cache entries are only returned in the case of an 500 and 504 (inclusive) status when revalidating a stale cache entry. We also made sure that the changes are consistent with both the classic and async APIs. Thank you again for your input was (Author: arturobernalg): Hi [~olegk] Thank you for taking the time to review my changes and providing feedback. You were right in pointing out that my initial implementation was not correct and could potentially lead to issues with the async protocol handling pipeline. After some further research , i made the necessary changes to ensure that stale cache entries are only returned in the case of an 500 and 504 (inclusive) status when revalidating a stale cache entry. We also made sure that the changes are consistent with both the classic and async APIs. Thank you again for your input > stale-if-error request cache directive should also apply to non-revalidatable > cache entries > --- > > Key: HTTPCLIENT-1368 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1368 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.4, 4.2.5, 4.3 Beta1 >Reporter: Jonathan Moore >Priority: Minor > Labels: stuck, volunteers-wanted > Fix For: Stuck > > > The stale-if-error request cache directive is defined in RFC 5861: > http://tools.ietf.org/html/rfc5861 > As implemented, this will only apply to cache entries that are revalidatable > (i.e. they have an Etag or Last-Modified header and can be refreshed with a > conditional request) but it should also apply to any stale cache entry. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1368) stale-if-error request cache directive should also apply to non-revalidatable cache entries
[ https://issues.apache.org/jira/browse/HTTPCLIENT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706602#comment-17706602 ] Arturo Bernal commented on HTTPCLIENT-1368: --- Hi [~olegk] Thank you for taking the time to review my changes and providing feedback. You were right in pointing out that my initial implementation was not correct and could potentially lead to issues with the async protocol handling pipeline. After some further research , i made the necessary changes to ensure that stale cache entries are only returned in the case of an 500 and 504 (inclusive) status when revalidating a stale cache entry. We also made sure that the changes are consistent with both the classic and async APIs. Thank you again for your input > stale-if-error request cache directive should also apply to non-revalidatable > cache entries > --- > > Key: HTTPCLIENT-1368 > URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1368 > Project: HttpComponents HttpClient > Issue Type: Improvement > Components: HttpCache >Affects Versions: 4.2.4, 4.2.5, 4.3 Beta1 >Reporter: Jonathan Moore >Priority: Minor > Labels: stuck, volunteers-wanted > Fix For: Stuck > > > The stale-if-error request cache directive is defined in RFC 5861: > http://tools.ietf.org/html/rfc5861 > As implemented, this will only apply to cache entries that are revalidatable > (i.e. they have an Etag or Last-Modified header and can be refreshed with a > conditional request) but it should also apply to any stale cache entry. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCLIENT-1368) stale-if-error request cache directive should also apply to non-revalidatable cache entries
[
https://issues.apache.org/jira/browse/HTTPCLIENT-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17699247#comment-17699247
]
Arturo Bernal commented on HTTPCLIENT-1368:
---
HI [~jonm] / [~olegk]
I've been working on implementing the stale-if-error request cache directive as
defined in RFC 5861 ([http://tools.ietf.org/html/rfc5861]) in the
BasicHttpCache class. So far, I've made the following changes:
* Added the private final CacheConfig cacheConfig and private final
CacheValidityPolicy cacheValidityPolicy fields to the BasicHttpCache class.
* Added the following code to the getCacheEntry method:
{code:java}
if (cacheConfig.isStaleIfErrorEnabled() &&
(cacheValidityPolicy.getStaleness(root,
Instant.now()).compareTo(TimeValue.ZERO_MILLISECONDS) > 0)) { return root;
}{code}
This will check if the staleIfError directive is enabled and if the root cache
entry is stale. If both conditions are met, it will return the stale cache
entry.
Additionally, I've also made a small change to the CacheConfig class to add a
setter method for the staleIfErrorEnabled field.
{code:java}
public Builder setStaleIfErrorEnabled(final boolean enabled) {
this.staleIfErrorEnabled = enabled;
return this;
}{code}
With these changes, we should now be able to properly implement the
stale-if-error request cache directive as defined in RFC 5861.
However, I'm not entirely sure if I'm going in the right direction with this,
so I would appreciate it if someone could take a look and provide some feedback.
> stale-if-error request cache directive should also apply to non-revalidatable
> cache entries
> ---
>
> Key: HTTPCLIENT-1368
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1368
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpCache
>Affects Versions: 4.2.4, 4.2.5, 4.3 Beta1
>Reporter: Jonathan Moore
>Priority: Minor
> Labels: stuck, volunteers-wanted
> Fix For: Stuck
>
>
> The stale-if-error request cache directive is defined in RFC 5861:
> http://tools.ietf.org/html/rfc5861
> As implemented, this will only apply to cache entries that are revalidatable
> (i.e. they have an Etag or Last-Modified header and can be refreshed with a
> conditional request) but it should also apply to any stale cache entry.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-478) Add "Forwarded" HttpRequestInterceptor
[ https://issues.apache.org/jira/browse/HTTPCORE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689983#comment-17689983 ] Arturo Bernal commented on HTTPCORE-478: HI [~olegk] I'll take a look deeper to the [rfc7239|https://www.rfc-editor.org/rfc/rfc7239.html] to make the interceptor according to it thank you > Add "Forwarded" HttpRequestInterceptor > -- > > Key: HTTPCORE-478 > URL: https://issues.apache.org/jira/browse/HTTPCORE-478 > Project: HttpComponents HttpCore > Issue Type: New Feature >Reporter: Gary D. Gregory >Priority: Major > > Add a "Forwarded" HttpRequestInterceptor to add the "Forwarded" HTTP header. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-674) InetAddressUtils scoped ID support
[
https://issues.apache.org/jira/browse/HTTPCORE-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687599#comment-17687599
]
Arturo Bernal commented on HTTPCORE-674:
[~michael-o] [~ckozak]
Can we close this ticker with this PR
[https://github.com/apache/httpcomponents-core/pull/388]
TH
> InetAddressUtils scoped ID support
> --
>
> Key: HTTPCORE-674
> URL: https://issues.apache.org/jira/browse/HTTPCORE-674
> Project: HttpComponents HttpCore
> Issue Type: Task
> Components: HttpCore
>Affects Versions: 5.1
>Reporter: Carter Kozak
>Priority: Major
> Fix For: 5.3-alpha1
>
>
> This is a remaining piece of
> https://issues.apache.org/jira/browse/HTTPCORE-616
> org.apache.hc.core5.net.InetAddressUtils: Parsing may fail when an IPv6 scope
> id might be provided.
> The following should likely succeed, but currently does not:
> {code:java}
> @Test
> public void testScopedIPv6Address() {
>
> Assert.assertTrue(InetAddressUtils.isIPv6Address("fe80::1ff:fe23:4567:890a"));
>
> Assert.assertTrue(InetAddressUtils.isIPv6Address("fe80::1ff:fe23:4567:890a%eth2"));
>
> Assert.assertTrue(InetAddressUtils.isIPv6Address("fe80::1ff:fe23:4567:890a%3"));
> }
> {code}
> This feature will impact parsing as well as detection, because as I
> understand it {{fe80::1ff:fe23:4567:890a%eth2"}} is URI encoded as
> {{https://[fe80::1ff:fe23:4567:890a%25eth2]:443/}} (note the {{%}} is
> percent-encoded to {{%25}}).
> This is based on a quick look at the ipv6 wiki page, I need to read though
> the current RFC guidance to verify the data I've provided is accurate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
[jira] [Commented] (HTTPCORE-479) Add "Via" HttpRequestInterceptor
[ https://issues.apache.org/jira/browse/HTTPCORE-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687254#comment-17687254 ] Arturo Bernal commented on HTTPCORE-479: Implemented in the Version 5.3 We can close this ticket [~olegk] > Add "Via" HttpRequestInterceptor > > > Key: HTTPCORE-479 > URL: https://issues.apache.org/jira/browse/HTTPCORE-479 > Project: HttpComponents HttpCore > Issue Type: New Feature >Reporter: Gary D. Gregory >Priority: Major > > Add a "Via" HttpRequestInterceptor to add the "Via" HTTP header. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org For additional commands, e-mail: dev-h...@hc.apache.org
