That would be great. There are many synchronous, threaded applications that want to ensure eventual completion and the work around is cumbersome and non-obvious.
It looks like there are already several related enhancement requests. Thanks again for the help. ________________________________ From: Michael McMahon <michael.x.mcma...@oracle.com> Sent: Thursday, October 7, 2021 7:40 AM To: Daniel Fuchs <daniel.fu...@oracle.com>; Elliot Barlas <elliot.bar...@logmein.com>; net-dev@openjdk.java.net <net-dev@openjdk.java.net> Subject: Re: HttpClient Send Method Guaranteed Completion I think we should at least clarify the behavior of HttpRequest.Builder.timeout() ie. that it is specifically related to the response headers, and maybe consider adding a timeout variant that sets the timeout for the entire response, including the body. - Michael. On 06/10/2021 10:42, Daniel Fuchs wrote: Hi Eliot, Unless the server keeps the connection open, and fails to send all the body bytes it has promised to send, the send operation should eventually terminate. The behavior you describe looks indeed like a bug. Could you please log a ticket with: https://bugreport.java.com/bugreport/<https://urldefense.com/v3/__https://bugreport.java.com/bugreport/__;!!OA8L0MA-!qbgY3D4kbHdhzZj7vTjF-gwr4hN9lyHcUoBPZW_5vUCebofdPVe937JEa-sJwAZmBNg$> That said - there is very little information here to work with. It would be helpful to understand with which version of the protocol this happened: HTTP or HTTPS? Version 1.1 or Version 2? Was it with an upgrade request (h2c)? If you manage to reproduce, it would be helpful if HTTP traces could be enabled - I'd suggest: `-Djdk.httpclient.HttpClient.log=headers,requests,errors` (see https://docs.oracle.com/en/java/javase/17/core/java-networking.html<https://urldefense.com/v3/__https://docs.oracle.com/en/java/javase/17/core/java-networking.html__;!!OA8L0MA-!qbgY3D4kbHdhzZj7vTjF-gwr4hN9lyHcUoBPZW_5vUCebofdPVe937JEa-sJ67i7iPI$>) Another issue here is that the request timeout set through `HttpRequestBuilder::timeout` only runs until the response headers are received. The reception of the body bytes is not covered by this timeout. This is not well documented, and we should probably do a better job there. We have an enhancement request related to this: https://bugs.openjdk.java.net/browse/JDK-8258397<https://urldefense.com/v3/__https://bugs.openjdk.java.net/browse/JDK-8258397__;!!OA8L0MA-!qbgY3D4kbHdhzZj7vTjF-gwr4hN9lyHcUoBPZW_5vUCebofdPVe937JEa-sJu5TSfSc$> However - setting up a global timeout for the request is already possible: If you wish to set a timeout for the reception of the body bytes you can do so by making use of the timeout facility provided by CompletableFuture: https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html#get(long,java.util.concurrent.TimeUnit)<https://urldefense.com/v3/__https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/concurrent/CompletableFuture.html*get(long,java.util.concurrent.TimeUnit)__;Iw!!OA8L0MA-!qbgY3D4kbHdhzZj7vTjF-gwr4hN9lyHcUoBPZW_5vUCebofdPVe937JEa-sJKoYa4r4$> Instead of: ``` String html = httpClient.send(request, HttpResponse.BodyHandlers.ofString()).body(); ``` Use: ``` long timeout = 5000; // ms String html; CompletableFuture<HttpResponse<String>> cf; try { cf = httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString()); html = cf.get(timeout, TimedUnit.MILLISECONDS) .body(); } catch (TimeoutException x) { // timeout: cancel request to free up // any related resources cf.cancel(); } catch (CancellationException x) { // cancelled } catch (InterruptedException x) { // interrupted } catch (ExecutionException x) { // IO etc.. // unwrap to get exception and rethrow... } ``` This offers the possibility to set up a global timeout You can also cancel the request asynchronously at any time by calling cf.cancel() (since Java 15 IIRC). best regards, -- daniel On 05/10/2021 23:05, Elliot Barlas wrote: Hello net-dev! I'm emailing about a surprising observation in the HttpClient send method related to guaranteed completion. Despite explicit timeout configurations, a call to send can block indefinitely. The stacktrace below was obtained from a thread dump on a running OpenJDK 16.0.2 JVM. The thread was stuck in that state for over a week. An application restart was required to recover. The related Java source code is also included below. My sense is that this is due to a missed notification from the async I/O subsystem. Unfortunately, I'm not an expert on HttpClient internals, and the asynchronous nature of the code makes it challenging to debug. Is this the intended behavior? That is, are there scenarios in which send should never return? Did I miss a socket timeout option? ----- HttpClient httpClient = HttpClient.newBuilder() .version(HttpClient.Version.HTTP_1_1) .connectTimeout(Duration.ofSeconds(5)) .build(); ... HttpRequest request = HttpRequest.newBuilder() .timeout(Duration.ofSeconds(5)) .uri(URI.create("http://";<https://urldefense.com/v3/__http://*22__;JQ!!OA8L0MA-!qbgY3D4kbHdhzZj7vTjF-gwr4hN9lyHcUoBPZW_5vUCebofdPVe937JEa-sJJqAshag$> + ip + ":" + port + "/frontend")) .build(); ... String html = httpClient.send(request, HttpResponse.BodyHandlers.ofString()).body(); ----- "pool-1-thread-70" #164 prio=5 os_prio=0 cpu=100038.29ms elapsed=1891832.58s tid=0x00007fe0211d4050 nid=0x29e waiting on condition [0x00007fdf025ac000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@16.0.2/Native<mailto:java.base@16.0.2/Native> Method) - parking to wait for <0x00000000e7fc9be0> (a java.util.concurrent.CompletableFuture$Signaller) at java.util.concurrent.locks.LockSupport.park(java.base@16.0.2/LockSupport.java:211<mailto:java.base@16.0.2/LockSupport.java:211>) at java.util.concurrent.CompletableFuture$Signaller.block(java.base@16.0.2/CompletableFuture.java:1860<mailto:java.base@16.0.2/CompletableFuture.java:1860>) at java.util.concurrent.ForkJoinPool.managedBlock(java.base@16.0.2/ForkJoinPool.java:3137<mailto:java.base@16.0.2/ForkJoinPool.java:3137>) at java.util.concurrent.CompletableFuture.waitingGet(java.base@16.0.2/CompletableFuture.java:1894<mailto:java.base@16.0.2/CompletableFuture.java:1894>) at java.util.concurrent.CompletableFuture.get(java.base@16.0.2/CompletableFuture.java:2068<mailto:java.base@16.0.2/CompletableFuture.java:2068>) at jdk.internal.net.http.HttpClientImpl.send(java.net.http@16.0.2/HttpClientImpl.java:535<mailto:java.net.http@16.0.2/HttpClientImpl.java:535>) at jdk.internal.net.http.HttpClientFacade.send(java.net.http@16.0.2/HttpClientFacade.java:119<mailto:java.net.http@16.0.2/HttpClientFacade.java:119>) at com.logmein.haproxy.ProxyFrontendConnectionCounter.countCurrentConnections(ProxyFrontendConnectionCounter.java:89) at com.logmein.haproxy.ProxyFrontendConnectionCounter.doRun(ProxyFrontendConnectionCounter.java:74) at com.logmein.haproxy.ProxyFrontendConnectionCounter.run(ProxyFrontendConnectionCounter.java:59) at com.logmein.haproxy.ProxyFrontendConnectionCounter$$Lambda$712/0x00000008010a0a88.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(java.base@16.0.2/Executors.java:515<mailto:java.base@16.0.2/Executors.java:515>) at java.util.concurrent.FutureTask.runAndReset(java.base@16.0.2/FutureTask.java:305<mailto:java.base@16.0.2/FutureTask.java:305>) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(java.base@16.0.2/ScheduledThreadPoolExecutor.java:305<mailto:java.base@16.0.2/ScheduledThreadPoolExecutor.java:305>) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@16.0.2/ThreadPoolExecutor.java:1130<mailto:java.base@16.0.2/ThreadPoolExecutor.java:1130>) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@16.0.2/ThreadPoolExecutor.java:630<mailto:java.base@16.0.2/ThreadPoolExecutor.java:630>) at java.lang.Thread.run(java.base@16.0.2/Thread.java:831<mailto:java.base@16.0.2/Thread.java:831>) Elliot Barlas elliot.bar...@logmein.com<mailto:elliot.bar...@logmein.com>
