How to reuse TCP connection when using a proxy with java.net.http.HttpClient in HTTP/1.1

[Nicolas Henneaux]

Hi,

I have a problem when using Java HttpClient with a proxy and HTTP/1.1
version. When using a stream body handler, there is no re-use of the TCP
connection.
When using another body handler
(java.net.http.HttpResponse.BodyHandlers#ofString()
or java.net.http.HttpResponse.BodyHandlers#discarding) there is no such
problem.
When using HTTP/2, there is no problem as well.

The following code reproduce the problem. The re-use can be checked
using *netstat
-tn  | grep '3128'  *
class ProxyHttp1Check {
    public static void main(String[] args) throws Exception {
        HttpClient httpClient = HttpClient.newBuilder()
            .version(HttpClient.Version.HTTP_1_1)
            .proxy(ProxySelector.of(InetSocketAddress.createUnresolved("
my-proxy.net", 3128)))
            .build();
        HttpRequest httpRequest = HttpRequest.newBuilder(URI.create("
https://my-website.test";)).build();
        for (int i = 0; i < 1_000_000_000; i++) {
            httpClient
                .sendAsync(httpRequest,
HttpResponse.BodyHandlers.ofInputStream())
                .get()
                .body()
                .close(); // It seems this is closing the TCP connection
        }
    }
}

The close comes from the client as underlined in the following dump.

T 2021/01/11 19:24:59.880557 client-ip:57422 -> proxy-ip:3128 [S] #265
T 2021/01/11 19:24:59.881067 proxy-ip:3128 -> client-ip:57422 [AS] #266
T 2021/01/11 19:24:59.881098 client-ip:57422 -> proxy-ip:3128 [A] #267
T 2021/01/11 19:24:59.881671 client-ip:57422 -> proxy-ip:3128 [AP] #268
  43 4f 4e 4e 45 43 54 20    61 70 69 2d 74 65 73 74    CONNECT api-test
  2e 61 77 73 2e 62 61 6d    62 6f 72 61 2e 63 6f 6d    .aws.bambora.com
  3a 34 34 33 20 48 54 54    50 2f 31 2e 31 0d 0a 0d    :443 HTTP/1.1...
  0a                                                    .
T 2021/01/11 19:24:59.882077 proxy-ip:3128 -> client-ip:57422 [A] #269
T 2021/01/11 19:24:59.883605 proxy-ip:3128 -> client-ip:57422 [AP] #270
  48 54 54 50 2f 31 2e 31    20 32 30 30 20 43 6f 6e    HTTP/1.1 200 Con
  6e 65 63 74 69 6f 6e 20    65 73 74 61 62 6c 69 73    nection establis
  68 65 64 0d 0a 0d 0a                                  hed....
T 2021/01/11 19:24:59.883627 client-ip:57422 -> proxy-ip:3128 [A] #271
T 2021/01/11 19:24:59.933932 client-ip:57422 -> proxy-ip:3128 [AP] #272
T 2021/01/11 19:24:59.952473 proxy-ip:3128 -> client-ip:57422 [AP] #273
T 2021/01/11 19:24:59.952515 client-ip:57422 -> proxy-ip:3128 [A] #274
*T 2021/01/11 19:25:00.298765 client-ip:57422 -> proxy-ip:3128 [AF] #275*
T 2021/01/11 19:25:00.299777 proxy-ip:3128 -> client-ip:57422 [AF] #276
T 2021/01/11 19:25:00.299809 client-ip:57422 -> proxy-ip:3128 [A] #277

I have tested it on Java 11.0.9.1, 14.0.2 and 15.0.1 on Ubuntu 20.4.1.

 $ /usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java -version
openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.9.1+1-Ubuntu-0ubuntu1.20.04, mixed
mode, sharing)
 $ java -version
openjdk version "14.0.2" 2020-07-14
OpenJDK Runtime Environment (build 14.0.2+12-Ubuntu-120.04)
OpenJDK 64-Bit Server VM (build 14.0.2+12-Ubuntu-120.04, mixed mode,
sharing)
 $ java -version
openjdk version "15.0.1" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.1+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.1+9, mixed mode, sharing)
$ uname -a
Linux enix-002 5.8.0-36-generic #40~20.04.1-Ubuntu SMP Wed Jan 6 10:15:55
UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Thanks in advance for your help,

Best regards,

Nicolas Henneaux