Thanks Oleg. The keyword “reset” is enough to help us point networking in a plausible direction. - since this is a symptom associated with a broker connection this confirms why each retry always fails within the same 5ms elapsed time; - we currently allow a retry using the same execution context; - obviously a fruitless waste of resource time;
— Pete On 5/6/16, 5:24 AM, "Oleg Kalnichevski" <ol...@apache.org> wrote: >On Thu, 2016-05-05 at 14:46 +0000, Pete Keyes wrote: >> Update for the elapsed time for these calls that fail strictly at >>startup. >> The stair-step 5/25/500... was strictly due to a pause we insert >>between >> retries. All 4 tries fail within 5ms. >> >> Given the configuration settings for CloseableHttpClient a failure >>within >> 5ms seems unlikely. The trip to the F5 VIP and back would barely >>complete >> within 5ms. >> >> >> Any thoughts on how to stabilize this a bit at startup? Or glean more >> information about why apache-hc would fail with NoHttpResponseException >>so >> quickly. > >The best course of action would be to capture IP packets with a traffic >analyzer such as Wireshark and to try to figure out why TCP connections >get reset. > >Oleg > > > >> Stack trace for the error: >> org.apache.http.NoHttpResponseException: >> crm-asp-prod.starbucks.net:443 failed to respond >> >> >>org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttp >>Re >> sponseParser.java:143) >> >> >>org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttp >>Re >> sponseParser.java:57) >> >> >>org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser >>.j >> ava:261) >> >> >>org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(D >>ef >> aultBHttpClientConnection.java:165) >> >> >>org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.jav >>a: >> 167) >> >> >>org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpReques >>tE >> xecutor.java:272) >> >> >>org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor. >>ja >> va:124) >> >> >>org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java >>:2 >> 71) >> >> >>org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184 >>) >> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) >> >> >>org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110 >>) >> >> >>org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClie >>nt >> .java:184) >> >> >>org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClie >>nt >> .java:82) >> >> >> >> ‹ >> Pete Keyes >> >> >> >> >> >> >> On 5/4/16, 11:13 AM, "Pete Keyes" <pke...@starbucks.com> wrote: >> >> >We have an odd issue wherein at Tomcat(TomEE) startup apache-hc >>execution >> >requests throw a NoHttpResponseException within less than 1 second. >> >After a few minutes this behavior completely stops and isn¹t seen again >> >until the next time we stop and restart Tomcat(TomEE). >> > >> >Basic apache-hc configuration information: >> > - single static CloseableHttpClient instance initialized at container >> >startup; >> > - all requests managed via PoolingHttpClientConnectionManager >> > - connection timeout 6000ms >> > - read timeout 6000ms >> > - we use 4 retries >> > - the typical timing for each >> >HttpClient.execute(HttpUriRequest,HttpContext) during this startup >>period >> >is that they fail very fast: >> > try-1: ~ 5ms >> > try-2: ~250ms >> > try-3: ~500ms >> > try-5: ~700ms >> >We measure the elapsed milli-seconds with respect to ³execute()². It >> >seems very odd that execute() fails so fast given the apache-hc >>settings. >> > It isn¹t every execution request that fails. But, a very high >> >percentage of them do exhibit this behavior during the initial ~5 >>minutes >> >after Tomcat(TomEE) begins dispatching inbound requests. >> > >> >The inbound HTTP request volume inbound is ~70 RPS(requests per >>second). >> > >> >After ~5 minutes this type of error never repeats. We have logging in >> >place that verifies all the apache-hc components are fully instantiated >> >prior to any request being processed. >> > >> >Version information: >> > apache-hc: 4.5.2 >> > java : 1.8r60 >> > tomcat : 7.0.68 >> > tomee : 1.7.4 >> > >> > >> >Any ideas at all given the minimal information that I have to provide >>are >> >greatly appreciated. >> > >> >‹ >> >Pete >> > >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org >> For additional commands, e-mail: httpclient-users-h...@hc.apache.org >> > > > >--------------------------------------------------------------------- >To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org >For additional commands, e-mail: httpclient-users-h...@hc.apache.org >