pooling connection manager: changing max per route
version: 4.4.1 App Server: TomEE7 We use PoolingHttpClientConnectionManager with the following defaults: * max total connections: 100,000 * default max per route: 50 we create the HttpClient and PoolingHttpClientConnectionManager as static singletons at container startup. private static Thread globalConnManagerThread; private static CloseableHttpClient globalHttpClient; static { // create global pooling connection manager globalConnManagerPool = new PoolingHttpClientConnectionManager( registry ); globalConnManagerPool.setDefaultMaxPerRoute(50); globalConnManagerPool.setMaxTotal(10); // create global HttpClient instance final HttpClientBuilder builder = HttpClients.custom() .setConnectionManager(globalConnManagerPool) .setConnectionManagerShared(true) .setConnectionTimeToLive(5, TimeUnit.MINUTES) .setDefaultConnectionConfig(connConfig) .setDefaultRequestConfig(requestConfig) .setMaxConnPerRoute(50) // <<== default per route is 50 .setDefaultSocketConfig(socketConfig) .setMaxConnTotal(10) .setRetryHandler(httpRetryHandler) .useSystemProperties() ; CloseableHttpClient globalHttpClient = builder.build(); } We set the max per specific route with the following code snippet: @PostConstruct public void init() { List allUrls = getAllUrls(); for(final URL url : allUrls) { HttpServiceConfig hsc = getHttpServiceConfig(url); final String host = url.getHost(); final int portNo = url.getPort() == -1 ? url.getDefaultPort() : url.getPort(); final String protocol = url.getProtocol(); final HttpHost httpHost = new HttpHost(host, portNo, protocol); final HttpRoute httpRoute = new HttpRoute(httpHost); globalConnManagerPool.setMaxPerRoute(httpRoute, 200); // <<== max set to 200 for route log.info(gMarker, "set route max-conns:" + "; max-allowed=" + hsc.getMaxConns() + "; protocol-host-port=" + protocol + "://" + host + ":" + portNo + "; httpRoute=" + String.valueOf(httpRoute) ); } } we see each route specific max connection log message at startup. for instance: set route max-conns: conns-max=200; protocol-host-port=https://some-host:443; httpRoute={}->https://some-host:443 we have a background thread that monitors the apache-hc connection pool statistics and emits log messages. we see the following apache-hc connection manager pool “route specific” statistics logged: private void logPoolStats() { Set routes = globalConnManagerPool.getRoutes(); for(final HttpRoute r : routes) { final PoolStats ps = cm.getStats(r); final int available = ps.getAvailable(); final int active = ps.getLeased(); final int limit = ps.getMax(); final int blocking = ps.getPending(); String s = "emit statistic:" + " type=hc-conn-pool" + "; route=" + hostname + "; available=" + available + "; active=" + active + "; blocking=" + blocking + "; max-allowed=" + limit + "; server-nm=" + Helpers.getLocalHost() + ":" + Helpers.getServerHttpsPort() ; log.info(marker, s); } } we see log messages like the following from the apache-hc connection pool monitor logging: emit statistic: type=hc-conn-pool; route=some-host; available=1; active=0; blocking=0; max-allowed=50 The connection manager isn’t respecting the per-route connection settings. Above we see that the “max-allowed” per route reported by the pool-manager is 50 for a route that specifically set the max-allowed to 200. Any hints about what we are doing wrong to override the limit on a per-route basis? — Pete Keyes
Re: HttpClient SSL Connection Issue
On Mon, 2016-02-22 at 15:08 -0500, Murat Balkan wrote: > Tom, > It worked like a charm! Thank you very much! It seems HTTPURLConnection > and URL classes add a default "accept" header which is not implemented by > Apache HttpClient. 'Accept' header as well as other content negotiation headers are optional. HTTP agents do not have to implement content negotiation. https://tools.ietf.org/html/rfc7231#section-5.3 Oleg > Thanks again, > Murat > > On Mon, Feb 22, 2016 at 3:03 PM, Tim Jacomb [DATACOM] > wrote: > > > Try adding an Accept Header, the server you are contacting appears to > > reject all requests without one > > > > httpGet.addHeader("Accept", > > "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"); > > > > Tim > > > > From: Murat Balkan > > Sent: Tuesday, 23 February 2016 8:38 a.m. > > To: HttpClient User Discussion > > Subject: Re: HttpClient SSL Connection Issue > > > > Hi, Please find it below: The version does not matter. Whatever version I > > tried failed. I even installed Java 8 to test. My current versions in the > > built path are: httpclient4.5.1,httpcore4.4.3,httpmime 4.5.1, jna 4.1.0, > > jna-platform4.1.0,httpclient-cache4.5.1 > > > > SSLContext sslContext = SSLContexts.createDefault(); > > SSLConnectionSocketFactory sslConnectionFactory = new > > SSLConnectionSocketFactory(sslContext,NoopHostnameVerifier.INSTANCE); > > Registry socketFactoryRegistry = > > RegistryBuilder.create() > > .register("http", PlainConnectionSocketFactory.getSocketFactory()) > > .register("https", sslConnectionFactory) > > .build(); > > PoolingHttpClientConnectionManager cm = new > > PoolingHttpClientConnectionManager(socketFactoryRegistry); > > cm.setDefaultMaxPerRoute(1); > > CloseableHttpClient httpClient = HttpClientBuilder.create() > > .disableContentCompression() > > .disableAutomaticRetries() > > .setUserAgent("User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) > > Gecko/20100101 Firefox/44.0") > > .build(); > > HttpGet httpGet = new HttpGet("https://so.n11.com";); > > httpClient.execute(httpGet); > > System.out.println("I can never reach this point"); > > > > On Mon, Feb 22, 2016 at 2:33 PM, Philippe Mouawad < > > philippe.moua...@gmail.com> wrote: > > > > > hi, > > > Can you show your httpclient code and mention which versions you used for > > > httpcore and httpclient and java exact version > > > > > > thx > > > > > > On Monday, February 22, 2016, Murat Balkan wrote: > > > > > > > I tried the following with URL class this time: > > > > > > > > URL my_url = new URL("https://so.n11.com";); > > > > BufferedReader br = new BufferedReader(new > > > > InputStreamReader(my_url.openStream())); > > > > String strTemp = ""; > > > > while(null != (strTemp = br.readLine())){ > > > > System.out.println(strTemp); > > > > } > > > > > > > > Even this works. So far URL, HttpURLConnection and Browsers are able to > > > > fetch the page. Apache HttpClient cannot. > > > > I also tried Fluent from the same package. It fails too. > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > > > > > > > wrote: > > > > > > > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > > > > Can you please suggest me how HttpClient can get the page like > > > > > > HttpUrlConnection? or Google Chrome? > > > > > > > > > > Capture a session between a browser or HttpUrlConnection using > > > Wireshark > > > > > or browser plugin of your choosing. Configure HttpClient to generate > > > > > identical messages. See what happens. > > > > > > > > > > > If I go to the site admin, wouldnt he say the site is totally > > > > reachable? > > > > > > > > > > > > > > > > The admin can say that Earth is flat for all I care, but their server > > > > > drops connections without sending back a status code like all well > > > > > behaved, spec complaint HTTP servers are supposed to do. > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski < > > > ol...@apache.org > > > > > > > > > > > wrote: > > > > > > > > > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > > > > > Hi Oleg, > > > > > > > > I do not aggree, other Http libraries does not have this > > problem. > > > > As > > > > > I > > > > > > > said > > > > > > > > HttpUrlConnection gets the page, all types of browsers can get > > > the > > > > > page. > > > > > > > It > > > > > > > > is clear that this is an error that is related with the Apache > > > > > Client. > > > > > > > > Thnaks > > > > > > > > Murat > > > > > > > > > > > > > > You are very welcome to disagree. > > > > > > > > > > > > > > Your own log clearly show that the problem has nothing to do with > > > SSL > > > > > > > and is caused by peer connection reset. > > > > > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:1
Re: HttpClient SSL Connection Issue
Hi Oleg, I was wrong as per my N-1 mail . On Mon, Feb 22, 2016 at 9:13 PM, Oleg Kalnichevski wrote: > On Mon, 2016-02-22 at 21:08 +0100, Philippe Mouawad wrote: > > Hi Oleg, > > I tried the URL using JMeter and HttpClient 4.5.2 it fails. > > > > I don't see big differences when comparing ssl logs: > > Thread Group 1-1, handling exception: java.net.SocketException: > Connection > > reset > > %% Invalidated: [Session-4, TLS_RSA_WITH_AES_128_CBC_SHA] > > Thread Group 1-1, SEND TLSv1 ALERT: fatal, description = > unexpected_message > > Padded plaintext before ENCRYPTION: len = 32 > > : 02 0A 75 43 41 2D 66 FE B7 2F 45 02 3C 21 E7 67 ..uCA-f../E. > 0010: 6B 9C 21 52 18 37 09 09 09 09 09 09 09 09 09 09 k.!R.7.. > > Thread Group 1-1, WRITE: TLSv1 Alert, length = 32 > > Thread Group 1-1, Exception sending alert: java.net.SocketException: > Broken > > pipe > > Thread Group 1-1, called closeSocket() > > Thread Group 1-1, called close() > > Thread Group 1-1, called closeInternal(true) > > > > > > It seems like a bug in HttpClient no ? > > > > How exactly can this be a bug in HttpClient? > > Oleg > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > -- Cordialement. Philippe Mouawad.
Re: HttpClient SSL Connection Issue
On Mon, 2016-02-22 at 21:08 +0100, Philippe Mouawad wrote: > Hi Oleg, > I tried the URL using JMeter and HttpClient 4.5.2 it fails. > > I don't see big differences when comparing ssl logs: > Thread Group 1-1, handling exception: java.net.SocketException: Connection > reset > %% Invalidated: [Session-4, TLS_RSA_WITH_AES_128_CBC_SHA] > Thread Group 1-1, SEND TLSv1 ALERT: fatal, description = unexpected_message > Padded plaintext before ENCRYPTION: len = 32 > : 02 0A 75 43 41 2D 66 FE B7 2F 45 02 3C 21 E7 67 ..uCA-f../E. 0010: 6B 9C 21 52 18 37 09 09 09 09 09 09 09 09 09 09 k.!R.7.. > Thread Group 1-1, WRITE: TLSv1 Alert, length = 32 > Thread Group 1-1, Exception sending alert: java.net.SocketException: Broken > pipe > Thread Group 1-1, called closeSocket() > Thread Group 1-1, called close() > Thread Group 1-1, called closeInternal(true) > > > It seems like a bug in HttpClient no ? > How exactly can this be a bug in HttpClient? Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpClient SSL Connection Issue
Good catch @Tim ! Indeed Java: Padded plaintext before ENCRYPTION: len = 176 : 47 45 54 20 2F 20 48 54 54 50 2F 31 2E 31 0D 0A GET / HTTP/1.1.. 0010: 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 Connection: keep 0020: 2D 61 6C 69 76 65 0D 0A 55 73 65 72 2D 41 67 65 -alive..User-Age 0030: 6E 74 3A 20 4A 61 76 61 2F 31 2E 38 2E 30 5F 34 nt: Java/1.8.0_4 0040: 35 0D 0A 48 6F 73 74 3A 20 73 6F 2E 6E 31 31 2E 5..Host: so.n11. 0050: 63 6F 6D 0D 0A 41 63 63 65 70 74 3A 20 74 65 78 com..Accept: tex 0060: 74 2F 68 74 6D 6C 2C 20 69 6D 61 67 65 2F 67 69 t/html, image/gi 0070: 66 2C 20 69 6D 61 67 65 2F 6A 70 65 67 2C 20 2A f, image/jpeg, * 0080: 3B 20 71 3D 2E 32 2C 20 2A 2F 2A 3B 20 71 3D 2E ; q=.2, */*; q=. 0090: 32 0D 0A 0D 0A 32 8B 87 7A BA 17 82 81 CD BB C5 22..z... 00A0: F8 E3 E0 C4 B1 53 A6 09 63 06 06 06 06 06 06 06 .S..c... Thread Group 1-1, WRITE: TLSv1 Application Data, length = 176 HttpClient: Padded plaintext before ENCRYPTION: len = 144 : 47 45 54 20 2F 20 48 54 54 50 2F 31 2E 31 0D 0A GET / HTTP/1.1.. 0010: 43 6F 6E 6E 65 63 74 69 6F 6E 3A 20 6B 65 65 70 Connection: keep 0020: 2D 61 6C 69 76 65 0D 0A 48 6F 73 74 3A 20 73 6F -alive..Host: so 0030: 2E 6E 31 31 2E 63 6F 6D 0D 0A 55 73 65 72 2D 41 .n11.com..User-A 0040: 67 65 6E 74 3A 20 41 70 61 63 68 65 2D 48 74 74 gent: Apache-Htt 0050: 70 43 6C 69 65 6E 74 2F 34 2E 35 2E 32 2D 53 4E pClient/4.5.2-SN 0060: 41 50 53 48 4F 54 20 28 4A 61 76 61 2F 31 2E 38 APSHOT (Java/1.8 0070: 2E 30 5F 34 35 29 0D 0A 0D 0A 68 D5 4F F4 33 6B .0_45)h.O.3k 0080: 1B 37 6F 3B CC 01 D2 D8 7F 95 02 FB 58 3A 01 01 .7o;X:.. Thread Group 1-1, WRITE: TLSv1 Application Data, length = 144 Learnt something today , thanks ! On Mon, Feb 22, 2016 at 9:03 PM, Tim Jacomb [DATACOM] wrote: > Try adding an Accept Header, the server you are contacting appears to > reject all requests without one > > httpGet.addHeader("Accept", > "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"); > > Tim > > From: Murat Balkan > Sent: Tuesday, 23 February 2016 8:38 a.m. > To: HttpClient User Discussion > Subject: Re: HttpClient SSL Connection Issue > > Hi, Please find it below: The version does not matter. Whatever version I > tried failed. I even installed Java 8 to test. My current versions in the > built path are: httpclient4.5.1,httpcore4.4.3,httpmime 4.5.1, jna 4.1.0, > jna-platform4.1.0,httpclient-cache4.5.1 > > SSLContext sslContext = SSLContexts.createDefault(); > SSLConnectionSocketFactory sslConnectionFactory = new > SSLConnectionSocketFactory(sslContext,NoopHostnameVerifier.INSTANCE); > Registry socketFactoryRegistry = > RegistryBuilder.create() > .register("http", PlainConnectionSocketFactory.getSocketFactory()) > .register("https", sslConnectionFactory) > .build(); > PoolingHttpClientConnectionManager cm = new > PoolingHttpClientConnectionManager(socketFactoryRegistry); > cm.setDefaultMaxPerRoute(1); > CloseableHttpClient httpClient = HttpClientBuilder.create() > .disableContentCompression() > .disableAutomaticRetries() > .setUserAgent("User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) > Gecko/20100101 Firefox/44.0") > .build(); > HttpGet httpGet = new HttpGet("https://so.n11.com";); > httpClient.execute(httpGet); > System.out.println("I can never reach this point"); > > On Mon, Feb 22, 2016 at 2:33 PM, Philippe Mouawad < > philippe.moua...@gmail.com> wrote: > > > hi, > > Can you show your httpclient code and mention which versions you used for > > httpcore and httpclient and java exact version > > > > thx > > > > On Monday, February 22, 2016, Murat Balkan wrote: > > > > > I tried the following with URL class this time: > > > > > > URL my_url = new URL("https://so.n11.com";); > > > BufferedReader br = new BufferedReader(new > > > InputStreamReader(my_url.openStream())); > > > String strTemp = ""; > > > while(null != (strTemp = br.readLine())){ > > > System.out.println(strTemp); > > > } > > > > > > Even this works. So far URL, HttpURLConnection and Browsers are able to > > > fetch the page. Apache HttpClient cannot. > > > I also tried Fluent from the same package. It fails too. > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > > > > > wrote: > > > > > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > > > Can you please suggest me how HttpClient can get the page like > > > > > HttpUrlConnection? or Google Chrome? > > > > > > > > Capture a session between a browser or HttpUrlConnection using > > Wireshark > > > > or browser plugin of your choosing. Configure HttpClient to generate > > > > identical messages. See what happens. > > > > > > > > > If I go to the site admin, wouldnt he say the site is totally > > > reachable? > > > > > > > > > > > > > The admin can say that Earth is flat for all I care, but their server > >
Re: HttpClient SSL Connection Issue
Hi Oleg, I tried the URL using JMeter and HttpClient 4.5.2 it fails. I don't see big differences when comparing ssl logs: Thread Group 1-1, handling exception: java.net.SocketException: Connection reset %% Invalidated: [Session-4, TLS_RSA_WITH_AES_128_CBC_SHA] Thread Group 1-1, SEND TLSv1 ALERT: fatal, description = unexpected_message Padded plaintext before ENCRYPTION: len = 32 : 02 0A 75 43 41 2D 66 FE B7 2F 45 02 3C 21 E7 67 ..uCA-f../E. wrote: > Hi, Please find it below: The version does not matter. Whatever version I > tried failed. I even installed Java 8 to test. My current versions in the > built path are: httpclient4.5.1,httpcore4.4.3,httpmime 4.5.1, jna 4.1.0, > jna-platform4.1.0,httpclient-cache4.5.1 > > SSLContext sslContext = SSLContexts.createDefault(); > SSLConnectionSocketFactory sslConnectionFactory = new > SSLConnectionSocketFactory(sslContext,NoopHostnameVerifier.INSTANCE); > Registry socketFactoryRegistry = > RegistryBuilder.create() > .register("http", PlainConnectionSocketFactory.getSocketFactory()) > .register("https", sslConnectionFactory) > .build(); > PoolingHttpClientConnectionManager cm = new > PoolingHttpClientConnectionManager(socketFactoryRegistry); > cm.setDefaultMaxPerRoute(1); > CloseableHttpClient httpClient = HttpClientBuilder.create() > .disableContentCompression() > .disableAutomaticRetries() > .setUserAgent("User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) > Gecko/20100101 Firefox/44.0") > .build(); > HttpGet httpGet = new HttpGet("https://so.n11.com";); > httpClient.execute(httpGet); > System.out.println("I can never reach this point"); > > On Mon, Feb 22, 2016 at 2:33 PM, Philippe Mouawad < > philippe.moua...@gmail.com> wrote: > > > hi, > > Can you show your httpclient code and mention which versions you used for > > httpcore and httpclient and java exact version > > > > thx > > > > On Monday, February 22, 2016, Murat Balkan wrote: > > > > > I tried the following with URL class this time: > > > > > > URL my_url = new URL("https://so.n11.com";); > > > BufferedReader br = new BufferedReader(new > > > InputStreamReader(my_url.openStream())); > > > String strTemp = ""; > > > while(null != (strTemp = br.readLine())){ > > > System.out.println(strTemp); > > > } > > > > > > Even this works. So far URL, HttpURLConnection and Browsers are able to > > > fetch the page. Apache HttpClient cannot. > > > I also tried Fluent from the same package. It fails too. > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > > > > > wrote: > > > > > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > > > Can you please suggest me how HttpClient can get the page like > > > > > HttpUrlConnection? or Google Chrome? > > > > > > > > Capture a session between a browser or HttpUrlConnection using > > Wireshark > > > > or browser plugin of your choosing. Configure HttpClient to generate > > > > identical messages. See what happens. > > > > > > > > > If I go to the site admin, wouldnt he say the site is totally > > > reachable? > > > > > > > > > > > > > The admin can say that Earth is flat for all I care, but their server > > > > drops connections without sending back a status code like all well > > > > behaved, spec complaint HTTP servers are supposed to do. > > > > > > > > Oleg > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski < > > ol...@apache.org > > > > > > > > > wrote: > > > > > > > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > > > > Hi Oleg, > > > > > > > I do not aggree, other Http libraries does not have this > problem. > > > As > > > > I > > > > > > said > > > > > > > HttpUrlConnection gets the page, all types of browsers can get > > the > > > > page. > > > > > > It > > > > > > > is clear that this is an error that is related with the Apache > > > > Client. > > > > > > > Thnaks > > > > > > > Murat > > > > > > > > > > > > You are very welcome to disagree. > > > > > > > > > > > > Your own log clearly show that the problem has nothing to do with > > SSL > > > > > > and is caused by peer connection reset. > > > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski < > > > > ol...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > > > > > I enabled the debug log and it seems the connection is > > > > established . > > > > > > Any > > > > > > > > > ideas? Attaching below: > > > > > > > > > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > > > > > DefaultHttpClientConnectionOperator - > > > > > > > > > Connection established 142.133.240.86:34018<-> > > > 176.41.133.12:443 > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - > > Executing > > > > > > request > > > > > > > > GET > > > > > > > > > / HTTP/1.1 > > > > > > > > > 2016/02/22 10:49:45:146 EST [D
Re: HttpClient SSL Connection Issue
Tom, It worked like a charm! Thank you very much! It seems HTTPURLConnection and URL classes add a default "accept" header which is not implemented by Apache HttpClient. Thanks again, Murat On Mon, Feb 22, 2016 at 3:03 PM, Tim Jacomb [DATACOM] wrote: > Try adding an Accept Header, the server you are contacting appears to > reject all requests without one > > httpGet.addHeader("Accept", > "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"); > > Tim > > From: Murat Balkan > Sent: Tuesday, 23 February 2016 8:38 a.m. > To: HttpClient User Discussion > Subject: Re: HttpClient SSL Connection Issue > > Hi, Please find it below: The version does not matter. Whatever version I > tried failed. I even installed Java 8 to test. My current versions in the > built path are: httpclient4.5.1,httpcore4.4.3,httpmime 4.5.1, jna 4.1.0, > jna-platform4.1.0,httpclient-cache4.5.1 > > SSLContext sslContext = SSLContexts.createDefault(); > SSLConnectionSocketFactory sslConnectionFactory = new > SSLConnectionSocketFactory(sslContext,NoopHostnameVerifier.INSTANCE); > Registry socketFactoryRegistry = > RegistryBuilder.create() > .register("http", PlainConnectionSocketFactory.getSocketFactory()) > .register("https", sslConnectionFactory) > .build(); > PoolingHttpClientConnectionManager cm = new > PoolingHttpClientConnectionManager(socketFactoryRegistry); > cm.setDefaultMaxPerRoute(1); > CloseableHttpClient httpClient = HttpClientBuilder.create() > .disableContentCompression() > .disableAutomaticRetries() > .setUserAgent("User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) > Gecko/20100101 Firefox/44.0") > .build(); > HttpGet httpGet = new HttpGet("https://so.n11.com";); > httpClient.execute(httpGet); > System.out.println("I can never reach this point"); > > On Mon, Feb 22, 2016 at 2:33 PM, Philippe Mouawad < > philippe.moua...@gmail.com> wrote: > > > hi, > > Can you show your httpclient code and mention which versions you used for > > httpcore and httpclient and java exact version > > > > thx > > > > On Monday, February 22, 2016, Murat Balkan wrote: > > > > > I tried the following with URL class this time: > > > > > > URL my_url = new URL("https://so.n11.com";); > > > BufferedReader br = new BufferedReader(new > > > InputStreamReader(my_url.openStream())); > > > String strTemp = ""; > > > while(null != (strTemp = br.readLine())){ > > > System.out.println(strTemp); > > > } > > > > > > Even this works. So far URL, HttpURLConnection and Browsers are able to > > > fetch the page. Apache HttpClient cannot. > > > I also tried Fluent from the same package. It fails too. > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > > > > > wrote: > > > > > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > > > Can you please suggest me how HttpClient can get the page like > > > > > HttpUrlConnection? or Google Chrome? > > > > > > > > Capture a session between a browser or HttpUrlConnection using > > Wireshark > > > > or browser plugin of your choosing. Configure HttpClient to generate > > > > identical messages. See what happens. > > > > > > > > > If I go to the site admin, wouldnt he say the site is totally > > > reachable? > > > > > > > > > > > > > The admin can say that Earth is flat for all I care, but their server > > > > drops connections without sending back a status code like all well > > > > behaved, spec complaint HTTP servers are supposed to do. > > > > > > > > Oleg > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski < > > ol...@apache.org > > > > > > > > > wrote: > > > > > > > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > > > > Hi Oleg, > > > > > > > I do not aggree, other Http libraries does not have this > problem. > > > As > > > > I > > > > > > said > > > > > > > HttpUrlConnection gets the page, all types of browsers can get > > the > > > > page. > > > > > > It > > > > > > > is clear that this is an error that is related with the Apache > > > > Client. > > > > > > > Thnaks > > > > > > > Murat > > > > > > > > > > > > You are very welcome to disagree. > > > > > > > > > > > > Your own log clearly show that the problem has nothing to do with > > SSL > > > > > > and is caused by peer connection reset. > > > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski < > > > > ol...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > > > > > I enabled the debug log and it seems the connection is > > > > established . > > > > > > Any > > > > > > > > > ideas? Attaching below: > > > > > > > > > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > > > > > DefaultHttpClientConnectionOperator - > > > > > > > > > Connection established 142.133.240.86:34018<-> > > > 176.41.133.
Re: HttpClient SSL Connection Issue
Try adding an Accept Header, the server you are contacting appears to reject all requests without one httpGet.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"); Tim From: Murat Balkan Sent: Tuesday, 23 February 2016 8:38 a.m. To: HttpClient User Discussion Subject: Re: HttpClient SSL Connection Issue Hi, Please find it below: The version does not matter. Whatever version I tried failed. I even installed Java 8 to test. My current versions in the built path are: httpclient4.5.1,httpcore4.4.3,httpmime 4.5.1, jna 4.1.0, jna-platform4.1.0,httpclient-cache4.5.1 SSLContext sslContext = SSLContexts.createDefault(); SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(sslContext,NoopHostnameVerifier.INSTANCE); Registry socketFactoryRegistry = RegistryBuilder.create() .register("http", PlainConnectionSocketFactory.getSocketFactory()) .register("https", sslConnectionFactory) .build(); PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(socketFactoryRegistry); cm.setDefaultMaxPerRoute(1); CloseableHttpClient httpClient = HttpClientBuilder.create() .disableContentCompression() .disableAutomaticRetries() .setUserAgent("User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0") .build(); HttpGet httpGet = new HttpGet("https://so.n11.com";); httpClient.execute(httpGet); System.out.println("I can never reach this point"); On Mon, Feb 22, 2016 at 2:33 PM, Philippe Mouawad < philippe.moua...@gmail.com> wrote: > hi, > Can you show your httpclient code and mention which versions you used for > httpcore and httpclient and java exact version > > thx > > On Monday, February 22, 2016, Murat Balkan wrote: > > > I tried the following with URL class this time: > > > > URL my_url = new URL("https://so.n11.com";); > > BufferedReader br = new BufferedReader(new > > InputStreamReader(my_url.openStream())); > > String strTemp = ""; > > while(null != (strTemp = br.readLine())){ > > System.out.println(strTemp); > > } > > > > Even this works. So far URL, HttpURLConnection and Browsers are able to > > fetch the page. Apache HttpClient cannot. > > I also tried Fluent from the same package. It fails too. > > > > > > > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > > > wrote: > > > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > > Can you please suggest me how HttpClient can get the page like > > > > HttpUrlConnection? or Google Chrome? > > > > > > Capture a session between a browser or HttpUrlConnection using > Wireshark > > > or browser plugin of your choosing. Configure HttpClient to generate > > > identical messages. See what happens. > > > > > > > If I go to the site admin, wouldnt he say the site is totally > > reachable? > > > > > > > > > > The admin can say that Earth is flat for all I care, but their server > > > drops connections without sending back a status code like all well > > > behaved, spec complaint HTTP servers are supposed to do. > > > > > > Oleg > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski < > ol...@apache.org > > > > > > > wrote: > > > > > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > > > Hi Oleg, > > > > > > I do not aggree, other Http libraries does not have this problem. > > As > > > I > > > > > said > > > > > > HttpUrlConnection gets the page, all types of browsers can get > the > > > page. > > > > > It > > > > > > is clear that this is an error that is related with the Apache > > > Client. > > > > > > Thnaks > > > > > > Murat > > > > > > > > > > You are very welcome to disagree. > > > > > > > > > > Your own log clearly show that the problem has nothing to do with > SSL > > > > > and is caused by peer connection reset. > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski < > > > ol...@apache.org > > > > > > > wrote: > > > > > > > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > > > > I enabled the debug log and it seems the connection is > > > established . > > > > > Any > > > > > > > > ideas? Attaching below: > > > > > > > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > > > > DefaultHttpClientConnectionOperator - > > > > > > > > Connection established 142.133.240.86:34018<-> > > 176.41.133.12:443 > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - > Executing > > > > > request > > > > > > > GET > > > > > > > > / HTTP/1.1 > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target > > auth > > > > > state: > > > > > > > > UNCHALLENGED > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy > auth > > > > > state: > > > > > > > > UNCHALLENGED > > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > > >> > > > GET > > > > > / > > >
Re: HttpClient SSL Connection Issue
Hi, Please find it below: The version does not matter. Whatever version I tried failed. I even installed Java 8 to test. My current versions in the built path are: httpclient4.5.1,httpcore4.4.3,httpmime 4.5.1, jna 4.1.0, jna-platform4.1.0,httpclient-cache4.5.1 SSLContext sslContext = SSLContexts.createDefault(); SSLConnectionSocketFactory sslConnectionFactory = new SSLConnectionSocketFactory(sslContext,NoopHostnameVerifier.INSTANCE); Registry socketFactoryRegistry = RegistryBuilder.create() .register("http", PlainConnectionSocketFactory.getSocketFactory()) .register("https", sslConnectionFactory) .build(); PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(socketFactoryRegistry); cm.setDefaultMaxPerRoute(1); CloseableHttpClient httpClient = HttpClientBuilder.create() .disableContentCompression() .disableAutomaticRetries() .setUserAgent("User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0") .build(); HttpGet httpGet = new HttpGet("https://so.n11.com";); httpClient.execute(httpGet); System.out.println("I can never reach this point"); On Mon, Feb 22, 2016 at 2:33 PM, Philippe Mouawad < philippe.moua...@gmail.com> wrote: > hi, > Can you show your httpclient code and mention which versions you used for > httpcore and httpclient and java exact version > > thx > > On Monday, February 22, 2016, Murat Balkan wrote: > > > I tried the following with URL class this time: > > > > URL my_url = new URL("https://so.n11.com";); > > BufferedReader br = new BufferedReader(new > > InputStreamReader(my_url.openStream())); > > String strTemp = ""; > > while(null != (strTemp = br.readLine())){ > > System.out.println(strTemp); > > } > > > > Even this works. So far URL, HttpURLConnection and Browsers are able to > > fetch the page. Apache HttpClient cannot. > > I also tried Fluent from the same package. It fails too. > > > > > > > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > > > wrote: > > > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > > Can you please suggest me how HttpClient can get the page like > > > > HttpUrlConnection? or Google Chrome? > > > > > > Capture a session between a browser or HttpUrlConnection using > Wireshark > > > or browser plugin of your choosing. Configure HttpClient to generate > > > identical messages. See what happens. > > > > > > > If I go to the site admin, wouldnt he say the site is totally > > reachable? > > > > > > > > > > The admin can say that Earth is flat for all I care, but their server > > > drops connections without sending back a status code like all well > > > behaved, spec complaint HTTP servers are supposed to do. > > > > > > Oleg > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski < > ol...@apache.org > > > > > > > wrote: > > > > > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > > > Hi Oleg, > > > > > > I do not aggree, other Http libraries does not have this problem. > > As > > > I > > > > > said > > > > > > HttpUrlConnection gets the page, all types of browsers can get > the > > > page. > > > > > It > > > > > > is clear that this is an error that is related with the Apache > > > Client. > > > > > > Thnaks > > > > > > Murat > > > > > > > > > > You are very welcome to disagree. > > > > > > > > > > Your own log clearly show that the problem has nothing to do with > SSL > > > > > and is caused by peer connection reset. > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski < > > > ol...@apache.org > > > > > > > wrote: > > > > > > > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > > > > I enabled the debug log and it seems the connection is > > > established . > > > > > Any > > > > > > > > ideas? Attaching below: > > > > > > > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > > > > DefaultHttpClientConnectionOperator - > > > > > > > > Connection established 142.133.240.86:34018<-> > > 176.41.133.12:443 > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - > Executing > > > > > request > > > > > > > GET > > > > > > > > / HTTP/1.1 > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target > > auth > > > > > state: > > > > > > > > UNCHALLENGED > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy > auth > > > > > state: > > > > > > > > UNCHALLENGED > > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > > >> > > > GET > > > > > / > > > > > > > > HTTP/1.1 > > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > > >> > > > > > Host: > > > > > > > > so.n11.com > > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > > >> > > > > > > > > Connection: Keep-Alive > > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > > >> > > > > > > > > User-Agent: Apache-HttpCl
Re: HttpClient SSL Connection Issue
hi, Can you show your httpclient code and mention which versions you used for httpcore and httpclient and java exact version thx On Monday, February 22, 2016, Murat Balkan wrote: > I tried the following with URL class this time: > > URL my_url = new URL("https://so.n11.com";); > BufferedReader br = new BufferedReader(new > InputStreamReader(my_url.openStream())); > String strTemp = ""; > while(null != (strTemp = br.readLine())){ > System.out.println(strTemp); > } > > Even this works. So far URL, HttpURLConnection and Browsers are able to > fetch the page. Apache HttpClient cannot. > I also tried Fluent from the same package. It fails too. > > > > > > On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski > > wrote: > > > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > > Can you please suggest me how HttpClient can get the page like > > > HttpUrlConnection? or Google Chrome? > > > > Capture a session between a browser or HttpUrlConnection using Wireshark > > or browser plugin of your choosing. Configure HttpClient to generate > > identical messages. See what happens. > > > > > If I go to the site admin, wouldnt he say the site is totally > reachable? > > > > > > > The admin can say that Earth is flat for all I care, but their server > > drops connections without sending back a status code like all well > > behaved, spec complaint HTTP servers are supposed to do. > > > > Oleg > > > > > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski > > > > wrote: > > > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > > Hi Oleg, > > > > > I do not aggree, other Http libraries does not have this problem. > As > > I > > > > said > > > > > HttpUrlConnection gets the page, all types of browsers can get the > > page. > > > > It > > > > > is clear that this is an error that is related with the Apache > > Client. > > > > > Thnaks > > > > > Murat > > > > > > > > You are very welcome to disagree. > > > > > > > > Your own log clearly show that the problem has nothing to do with SSL > > > > and is caused by peer connection reset. > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski < > > ol...@apache.org > > > > > > wrote: > > > > > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > > > I enabled the debug log and it seems the connection is > > established . > > > > Any > > > > > > > ideas? Attaching below: > > > > > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > > > DefaultHttpClientConnectionOperator - > > > > > > > Connection established 142.133.240.86:34018<-> > 176.41.133.12:443 > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing > > > > request > > > > > > GET > > > > > > > / HTTP/1.1 > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target > auth > > > > state: > > > > > > > UNCHALLENGED > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth > > > > state: > > > > > > > UNCHALLENGED > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > >> > > GET > > > > / > > > > > > > HTTP/1.1 > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > >> > > > > Host: > > > > > > > so.n11.com > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > >> > > > > > > > Connection: Keep-Alive > > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 > >> > > > > > > > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > > > > > > > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 > >> > > > > > > > Accept-Encoding: gzip,deflate > > > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > > > > DefaultManagedHttpClientConnection - > > > > > > > http-outgoing-3: Close connection > > > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > > > > DefaultManagedHttpClientConnection - > > > > > > > http-outgoing-3: Shutdown connection > > > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection > > > > discarded > > > > > > > > > > > > > > > > > > > The connection is dropped by the server due to an internal error > of > > > > some > > > > > > sort. You need to take it up with the server admin. > > > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > - > > > > > > To unsubscribe, e-mail: > httpclient-users-unsubscr...@hc.apache.org > > > > > > For additional commands, e-mail: > > httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - > > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > > > > > > > > > > - > > To unsubscr
Re: HttpClient SSL Connection Issue
I tried the following with URL class this time: URL my_url = new URL("https://so.n11.com";); BufferedReader br = new BufferedReader(new InputStreamReader(my_url.openStream())); String strTemp = ""; while(null != (strTemp = br.readLine())){ System.out.println(strTemp); } Even this works. So far URL, HttpURLConnection and Browsers are able to fetch the page. Apache HttpClient cannot. I also tried Fluent from the same package. It fails too. On Mon, Feb 22, 2016 at 12:40 PM, Oleg Kalnichevski wrote: > On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > > Can you please suggest me how HttpClient can get the page like > > HttpUrlConnection? or Google Chrome? > > Capture a session between a browser or HttpUrlConnection using Wireshark > or browser plugin of your choosing. Configure HttpClient to generate > identical messages. See what happens. > > > If I go to the site admin, wouldnt he say the site is totally reachable? > > > > The admin can say that Earth is flat for all I care, but their server > drops connections without sending back a status code like all well > behaved, spec complaint HTTP servers are supposed to do. > > Oleg > > > > > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski > > wrote: > > > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > > Hi Oleg, > > > > I do not aggree, other Http libraries does not have this problem. As > I > > > said > > > > HttpUrlConnection gets the page, all types of browsers can get the > page. > > > It > > > > is clear that this is an error that is related with the Apache > Client. > > > > Thnaks > > > > Murat > > > > > > You are very welcome to disagree. > > > > > > Your own log clearly show that the problem has nothing to do with SSL > > > and is caused by peer connection reset. > > > > > > Oleg > > > > > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski < > ol...@apache.org> > > > > wrote: > > > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > > I enabled the debug log and it seems the connection is > established . > > > Any > > > > > > ideas? Attaching below: > > > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > > DefaultHttpClientConnectionOperator - > > > > > > Connection established 142.133.240.86:34018<->176.41.133.12:443 > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing > > > request > > > > > GET > > > > > > / HTTP/1.1 > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth > > > state: > > > > > > UNCHALLENGED > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth > > > state: > > > > > > UNCHALLENGED > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > GET > > > / > > > > > > HTTP/1.1 > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > Host: > > > > > > so.n11.com > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > > > > Connection: Keep-Alive > > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > > > > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > > > > > > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> > > > > > > Accept-Encoding: gzip,deflate > > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > > > DefaultManagedHttpClientConnection - > > > > > > http-outgoing-3: Close connection > > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > > > DefaultManagedHttpClientConnection - > > > > > > http-outgoing-3: Shutdown connection > > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection > > > discarded > > > > > > > > > > > > > > > > The connection is dropped by the server due to an internal error of > > > some > > > > > sort. You need to take it up with the server admin. > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > - > > > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > > > For additional commands, e-mail: > httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > -- Murat Balkan
Re: HttpClient SSL Connection Issue
On Mon, 2016-02-22 at 11:27 -0500, Murat Balkan wrote: > Can you please suggest me how HttpClient can get the page like > HttpUrlConnection? or Google Chrome? Capture a session between a browser or HttpUrlConnection using Wireshark or browser plugin of your choosing. Configure HttpClient to generate identical messages. See what happens. > If I go to the site admin, wouldnt he say the site is totally reachable? > The admin can say that Earth is flat for all I care, but their server drops connections without sending back a status code like all well behaved, spec complaint HTTP servers are supposed to do. Oleg > On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski > wrote: > > > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > > Hi Oleg, > > > I do not aggree, other Http libraries does not have this problem. As I > > said > > > HttpUrlConnection gets the page, all types of browsers can get the page. > > It > > > is clear that this is an error that is related with the Apache Client. > > > Thnaks > > > Murat > > > > You are very welcome to disagree. > > > > Your own log clearly show that the problem has nothing to do with SSL > > and is caused by peer connection reset. > > > > Oleg > > > > > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski > > > wrote: > > > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > > I enabled the debug log and it seems the connection is established . > > Any > > > > > ideas? Attaching below: > > > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > > DefaultHttpClientConnectionOperator - > > > > > Connection established 142.133.240.86:34018<->176.41.133.12:443 > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing > > request > > > > GET > > > > > / HTTP/1.1 > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth > > state: > > > > > UNCHALLENGED > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth > > state: > > > > > UNCHALLENGED > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> GET > > / > > > > > HTTP/1.1 > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > Host: > > > > > so.n11.com > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > > > Connection: Keep-Alive > > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > > > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > > > > > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> > > > > > Accept-Encoding: gzip,deflate > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > > DefaultManagedHttpClientConnection - > > > > > http-outgoing-3: Close connection > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > > DefaultManagedHttpClientConnection - > > > > > http-outgoing-3: Shutdown connection > > > > > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection > > discarded > > > > > > > > > > > > > The connection is dropped by the server due to an internal error of > > some > > > > sort. You need to take it up with the server admin. > > > > > > > > Oleg > > > > > > > > > > > > - > > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > > > > > > > > > - > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpClient SSL Connection Issue
Can you please suggest me how HttpClient can get the page like HttpUrlConnection? or Google Chrome? If I go to the site admin, wouldnt he say the site is totally reachable? On Mon, Feb 22, 2016 at 11:22 AM, Oleg Kalnichevski wrote: > On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > > Hi Oleg, > > I do not aggree, other Http libraries does not have this problem. As I > said > > HttpUrlConnection gets the page, all types of browsers can get the page. > It > > is clear that this is an error that is related with the Apache Client. > > Thnaks > > Murat > > You are very welcome to disagree. > > Your own log clearly show that the problem has nothing to do with SSL > and is caused by peer connection reset. > > Oleg > > > > > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski > > wrote: > > > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > > I enabled the debug log and it seems the connection is established . > Any > > > > ideas? Attaching below: > > > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] > DefaultHttpClientConnectionOperator - > > > > Connection established 142.133.240.86:34018<->176.41.133.12:443 > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing > request > > > GET > > > > / HTTP/1.1 > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth > state: > > > > UNCHALLENGED > > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth > state: > > > > UNCHALLENGED > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> GET > / > > > > HTTP/1.1 > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > Host: > > > > so.n11.com > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > > Connection: Keep-Alive > > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > > > > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> > > > > Accept-Encoding: gzip,deflate > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > DefaultManagedHttpClientConnection - > > > > http-outgoing-3: Close connection > > > > 2016/02/22 10:49:45:419 EST [DEBUG] > DefaultManagedHttpClientConnection - > > > > http-outgoing-3: Shutdown connection > > > > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection > discarded > > > > > > > > > > The connection is dropped by the server due to an internal error of > some > > > sort. You need to take it up with the server admin. > > > > > > Oleg > > > > > > > > > - > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > -- Murat Balkan
Re: putRequest outputStream
On Mon, 2016-02-22 at 08:23 -0700, pavel wrote: > Hi, Oleg. > > I find can upload data if return from: > @Override > public long getContentLength() { > return -1; > } > some data length, instead of -1. Now I find walk around, though I think will > use HttpClient further and logging option will be useful. > Most likely the server can't handle chunk coded transfer encoding. Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpClient SSL Connection Issue
On Mon, 2016-02-22 at 11:18 -0500, Murat Balkan wrote: > Hi Oleg, > I do not aggree, other Http libraries does not have this problem. As I said > HttpUrlConnection gets the page, all types of browsers can get the page. It > is clear that this is an error that is related with the Apache Client. > Thnaks > Murat You are very welcome to disagree. Your own log clearly show that the problem has nothing to do with SSL and is caused by peer connection reset. Oleg > > On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski > wrote: > > > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > > I enabled the debug log and it seems the connection is established . Any > > > ideas? Attaching below: > > > > > > 2016/02/22 10:49:45:146 EST [DEBUG] DefaultHttpClientConnectionOperator - > > > Connection established 142.133.240.86:34018<->176.41.133.12:443 > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing request > > GET > > > / HTTP/1.1 > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth state: > > > UNCHALLENGED > > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth state: > > > UNCHALLENGED > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> GET / > > > HTTP/1.1 > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> Host: > > > so.n11.com > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > Connection: Keep-Alive > > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > > > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> > > > Accept-Encoding: gzip,deflate > > > 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - > > > http-outgoing-3: Close connection > > > 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - > > > http-outgoing-3: Shutdown connection > > > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection discarded > > > > > > > The connection is dropped by the server due to an internal error of some > > sort. You need to take it up with the server admin. > > > > Oleg > > > > > > - > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpClient SSL Connection Issue
Hi Oleg, I do not aggree, other Http libraries does not have this problem. As I said HttpUrlConnection gets the page, all types of browsers can get the page. It is clear that this is an error that is related with the Apache Client. Thnaks Murat On Mon, Feb 22, 2016 at 11:14 AM, Oleg Kalnichevski wrote: > On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > > I enabled the debug log and it seems the connection is established . Any > > ideas? Attaching below: > > > > 2016/02/22 10:49:45:146 EST [DEBUG] DefaultHttpClientConnectionOperator - > > Connection established 142.133.240.86:34018<->176.41.133.12:443 > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing request > GET > > / HTTP/1.1 > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth state: > > UNCHALLENGED > > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth state: > > UNCHALLENGED > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> GET / > > HTTP/1.1 > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> Host: > > so.n11.com > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > Connection: Keep-Alive > > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> > > Accept-Encoding: gzip,deflate > > 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - > > http-outgoing-3: Close connection > > 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - > > http-outgoing-3: Shutdown connection > > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection discarded > > > > The connection is dropped by the server due to an internal error of some > sort. You need to take it up with the server admin. > > Oleg > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > -- Murat Balkan
Re: putRequest outputStream
Hi, Oleg. I find can upload data if return from: @Override public long getContentLength() { return -1; } some data length, instead of -1. Now I find walk around, though I think will use HttpClient further and logging option will be useful. Thanks. -- View this message in context: http://httpcomponents.10934.n7.nabble.com/putRequest-outputStream-tp27956p27963.html Sent from the HttpClient-User mailing list archive at Nabble.com. - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpClient SSL Connection Issue
On Mon, 2016-02-22 at 10:57 -0500, Murat Balkan wrote: > I enabled the debug log and it seems the connection is established . Any > ideas? Attaching below: > > 2016/02/22 10:49:45:146 EST [DEBUG] DefaultHttpClientConnectionOperator - > Connection established 142.133.240.86:34018<->176.41.133.12:443 > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing request GET > / HTTP/1.1 > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth state: > UNCHALLENGED > 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth state: > UNCHALLENGED > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> GET / > HTTP/1.1 > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> Host: > so.n11.com > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > Connection: Keep-Alive > 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> > User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) > 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> > Accept-Encoding: gzip,deflate > 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - > http-outgoing-3: Close connection > 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - > http-outgoing-3: Shutdown connection > 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection discarded > The connection is dropped by the server due to an internal error of some sort. You need to take it up with the server admin. Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
Re: HttpClient SSL Connection Issue
I enabled the debug log and it seems the connection is established . Any ideas? Attaching below: 2016/02/22 10:49:45:146 EST [DEBUG] DefaultHttpClientConnectionOperator - Connection established 142.133.240.86:34018<->176.41.133.12:443 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Executing request GET / HTTP/1.1 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Target auth state: UNCHALLENGED 2016/02/22 10:49:45:146 EST [DEBUG] MainClientExec - Proxy auth state: UNCHALLENGED 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> GET / HTTP/1.1 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> Host: so.n11.com 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> Connection: Keep-Alive 2016/02/22 10:49:45:147 EST [DEBUG] headers - http-outgoing-3 >> User-Agent: Apache-HttpClient/4.5.1 (Java/1.7.0_79) 2016/02/22 10:49:45:148 EST [DEBUG] headers - http-outgoing-3 >> Accept-Encoding: gzip,deflate 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-3: Close connection 2016/02/22 10:49:45:419 EST [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-3: Shutdown connection 2016/02/22 10:49:45:419 EST [DEBUG] MainClientExec - Connection discarded On Sat, Feb 20, 2016 at 10:51 PM, Murat Balkan wrote: > Hi, > > I have a problem with HttpClient. (All versions, seems to have the same) > > When I try to connect an Https site (specifically so.n11.com) I got a > connection reset error after the handshake is finalized. If I try to call > the same URL with HttpUrlConnection, I dont get any errors. The browsers do > not have any problems displaying this site. > > I started thinking that this could be a bug, or I am doing something > wrong. I hope somebody can recognize this issue. > > > The code I am running is pretty straightforward: The same code works for > other HTTPS sites I tested. > > SSLConnectionSocketFactory sslConnectionFactory = new >> SSLConnectionSocketFactory(sslContext,new String[] >> {"TLSv1","TLSv1.1","TLSv1.2"},null, NoopHostnameVerifier.INSTANCE); >> Registry socketFactoryRegistry = >> RegistryBuilder.create() >> .register("http", PlainConnectionSocketFactory.getSocketFactory()) >> .register("https", sslConnectionFactory) >> .build(); >> PoolingHttpClientConnectionManager cm = new >> PoolingHttpClientConnectionManager(socketFactoryRegistry); >> cm.setDefaultMaxPerRoute(1); >> CloseableHttpClient httpClient = HttpClientBuilder.create().build(); >> HttpGet httpGet = new HttpGet("https://so.n11.com";); >> httpClient.execute(httpGet); >> System.out.println("I can never reach this point"); > > > > The exception I am receiving is: > > java.net.SocketException: Connection reset >> at java.net.SocketInputStream.read(Unknown Source) >> at java.net.SocketInputStream.read(Unknown Source) >> at sun.security.ssl.InputRecord.readFully(Unknown Source) >> at sun.security.ssl.InputRecord.read(Unknown Source) >> at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source) >> at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source) >> at sun.security.ssl.AppInputStream.read(Unknown Source) >> at >> org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) >> at >> org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) >> at >> org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) >> at >> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140) >> at >> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) >> at >> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) >> at >> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) >> at >> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167) >> at >> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) >> at >> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) >> at >> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271) >> at >> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) >> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) >> at >> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) >> at >> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) >> at HttpTest.main(HttpTest.java:102) > > > > My ssl debugged console output, The last line shows where it is crashing. > > > keyStore is : > keyStore type is : jks > keyStore provider is : > init keystore > init keymanager of type SunX509 > trustSto
Re: putRequest outputStream
On Sun, 2016-02-21 at 15:41 -0700, pavel wrote: > Hello > > I am new with CloseableHttpClient and HttpPut and want to use them, because > with java core HttpUrlConnection class I have to fully populate OutputStream > with data, before it starts transfer to server. While my data to send size > is big, this is not fits my need, because takes too much RAM. Data > collection speed much less than upload speed to server, where I want to PUT > data, so I want to use HttpPut with output stream. I was able to swithch > from Java core class and make PUT request with CloseableHttpClient and > HttpPut but only with short String data by ByteArrayEntity. When I try to > PUT with OutputStream, I get no errors/exceptions, but data entity not > transferred. After run I see on target server updated file, with zero > length. My network Interface traffic very small and looks like no entity > data transferred. You can see what gets written to the underlying socket by turning on wire logging as described here. http://hc.apache.org/httpcomponents-client-4.5.x/logging.html Oleg - To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org