Re: NoHttpResponseException received at startup

2016-05-06 Thread Oleg Kalnichevski 
On Thu, 2016-05-05 at 14:46 +, Pete Keyes wrote:
> Update for the elapsed time for these calls that fail strictly at startup.
>  The stair-step 5/25/500... was strictly due to a pause we insert between
> retries.  All 4 tries fail within 5ms.
> 
> Given the configuration settings for CloseableHttpClient a failure within
> 5ms seems unlikely.  The trip to the F5 VIP and back would barely complete
> within 5ms.
> 
> 
> Any thoughts on how to stabilize this a bit at startup?  Or glean more
> information about why apache-hc would fail with NoHttpResponseException so
> quickly.  

The best course of action would be to capture IP packets with a traffic
analyzer such as Wireshark and to try to figure out why TCP connections
get reset.

Oleg 



> Stack trace for the error:
> org.apache.http.NoHttpResponseException:
> crm-asp-prod.starbucks.net:443 failed to respond
> 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpRe
> sponseParser.java:143)
> 
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpRe
> sponseParser.java:57)
> 
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.j
> ava:261)
> 
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(Def
> aultBHttpClientConnection.java:165)
> 
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:
> 167)
> 
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestE
> xecutor.java:272)
> 
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.ja
> va:124)
> 
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:2
> 71)
> 
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
> org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
> 
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> 
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient
> .java:184)
> 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient
> .java:82)
> 
> 
> 
> ‹ 
> Pete Keyes
> 
> 
> 
> 
> 
> 
> On 5/4/16, 11:13 AM, "Pete Keyes"  wrote:
> 
> >We have an odd issue wherein at Tomcat(TomEE) startup apache-hc execution
> >requests throw a NoHttpResponseException within less than 1 second.
> >After a few minutes this behavior completely stops and isn¹t seen again
> >until the next time we stop and restart Tomcat(TomEE).
> >
> >Basic apache-hc configuration information:
> >  - single static CloseableHttpClient instance initialized at container
> >startup;
> >  - all requests managed via PoolingHttpClientConnectionManager
> >  - connection timeout 6000ms
> >  - read timeout 6000ms
> >  - we use 4 retries
> >  - the typical timing for each
> >HttpClient.execute(HttpUriRequest,HttpContext) during this startup period
> >is that they fail very fast:
> >  try-1: ~  5ms
> >  try-2: ~250ms
> >  try-3: ~500ms
> >  try-5: ~700ms
> >We measure the elapsed milli-seconds with respect to ³execute()².  It
> >seems very odd that execute() fails so fast given the apache-hc settings.
> > It isn¹t every execution request that fails.  But, a very high
> >percentage of them do exhibit this behavior during the initial ~5 minutes
> >after Tomcat(TomEE) begins dispatching inbound requests.
> >
> >The inbound HTTP request volume inbound is ~70 RPS(requests per second).
> >
> >After ~5 minutes this type of error never repeats.  We have logging in
> >place that verifies all the apache-hc components are fully instantiated
> >prior to any request being processed.
> >
> >Version information:
> >  apache-hc: 4.5.2
> >  java : 1.8r60
> >  tomcat   : 7.0.68
> >  tomee: 1.7.4
> >
> >
> >Any ideas at all given the minimal information that I have to provide are
> >greatly appreciated.
> >
> >‹
> >Pete
> >
> 
> 
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
> 



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org

Re: NoHttpResponseException received at startup

2016-05-05 Thread Pete Keyes 
Update for the elapsed time for these calls that fail strictly at startup.
 The stair-step 5/25/500... was strictly due to a pause we insert between
retries.  All 4 tries fail within 5ms.

Given the configuration settings for CloseableHttpClient a failure within
5ms seems unlikely.  The trip to the F5 VIP and back would barely complete
within 5ms.


Any thoughts on how to stabilize this a bit at startup?  Or glean more
information about why apache-hc would fail with NoHttpResponseException so
quickly.  Stack trace for the error:
org.apache.http.NoHttpResponseException:
crm-asp-prod.starbucks.net:443 failed to respond

org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpRe
sponseParser.java:143)

org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpRe
sponseParser.java:57)

org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.j
ava:261)

org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(Def
aultBHttpClientConnection.java:165)

org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:
167)

org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestE
xecutor.java:272)

org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.ja
va:124)

org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:2
71)

org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)

org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)

org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient
.java:184)

org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient
.java:82)



‹ 
Pete Keyes






On 5/4/16, 11:13 AM, "Pete Keyes"  wrote:

>We have an odd issue wherein at Tomcat(TomEE) startup apache-hc execution
>requests throw a NoHttpResponseException within less than 1 second.
>After a few minutes this behavior completely stops and isn¹t seen again
>until the next time we stop and restart Tomcat(TomEE).
>
>Basic apache-hc configuration information:
>  - single static CloseableHttpClient instance initialized at container
>startup;
>  - all requests managed via PoolingHttpClientConnectionManager
>  - connection timeout 6000ms
>  - read timeout 6000ms
>  - we use 4 retries
>  - the typical timing for each
>HttpClient.execute(HttpUriRequest,HttpContext) during this startup period
>is that they fail very fast:
>  try-1: ~  5ms
>  try-2: ~250ms
>  try-3: ~500ms
>  try-5: ~700ms
>We measure the elapsed milli-seconds with respect to ³execute()².  It
>seems very odd that execute() fails so fast given the apache-hc settings.
> It isn¹t every execution request that fails.  But, a very high
>percentage of them do exhibit this behavior during the initial ~5 minutes
>after Tomcat(TomEE) begins dispatching inbound requests.
>
>The inbound HTTP request volume inbound is ~70 RPS(requests per second).
>
>After ~5 minutes this type of error never repeats.  We have logging in
>place that verifies all the apache-hc components are fully instantiated
>prior to any request being processed.
>
>Version information:
>  apache-hc: 4.5.2
>  java : 1.8r60
>  tomcat   : 7.0.68
>  tomee: 1.7.4
>
>
>Any ideas at all given the minimal information that I have to provide are
>greatly appreciated.
>
>‹
>Pete
>


-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org