HttpClient performance question

2013-07-03 Thread Ke Ren
Hi, We are using httpcomponents httpclient 4.2.2 and doing loadtest with client. when we increased concurrent requests to 3000 per second, we found it took near half of cpu usage on aws ec m3.xlarge instance. We have config as the following: keepAlive is enabled. socket buffer size is 8 * 1024. m

Re: HttpClient performance question

2013-07-03 Thread Oleg Kalnichevski
On Wed, 2013-07-03 at 18:59 +0100, Ke Ren wrote: > Hi, > > We are using httpcomponents httpclient 4.2.2 and doing loadtest with > client. when we increased concurrent requests to 3000 per second, we found > it took near half of cpu usage on aws ec m3.xlarge instance. We have config > as the follow

Re: HttpClient performance question

2013-07-03 Thread Dave Johnston
Could you share the code ? Dave Johnston On 3 Jul 2013, at 19:00, "Ke Ren" wrote: > Hi, > > We are using httpcomponents httpclient 4.2.2 and doing loadtest with > client. when we increased concurrent requests to 3000 per second, we found > it took near half of cpu usage on aws ec m3.xlarge ins

Re: HttpClient performance question

2013-07-04 Thread Ke Ren
Thanks Oleg. 4.2 benchmark code is super useful. I ran this code there is no big difference between 4.3 and 4.2. I found STALE_CONNECTION_CHECK is false in benchmark test. If it's on it will drop Requests per second from 17k to 14k. I plugin our http client wrapper on top of apache http client in

Re: HttpClient performance question

2013-07-04 Thread Oleg Kalnichevski
On Thu, 2013-07-04 at 11:19 +0100, Ke Ren wrote: > Thanks Oleg. 4.2 benchmark code is super useful. I ran this code there is > no big difference between 4.3 and 4.2. I found STALE_CONNECTION_CHECK is > false in benchmark test. If it's on it will drop Requests per second from > 17k to 14k. > Yes,

Re: HttpClient performance question

2013-07-04 Thread Ke Ren
I will try keepAlive off with stale check off. we have the same http client usage like the one in benchmark code. that is only one client with pool manager shared by all requests. On Thu, Jul 4, 2013 at 11:41 AM, Oleg Kalnichevski wrote: > On Thu, 2013-07-04 at 11:19 +0100, Ke Ren wrote: > > Th

Re: HttpClient performance question

2013-07-04 Thread Ke Ren
I still get NoHttpResponseException after I turned off keepAlive and stale check. Do you mean closing connection every time after a request? I feel it is more expensive. Besides, the second call was sent now immediately after the first call was replied in milliseconds. I can't see any long idle in

Re: HttpClient performance question

2013-07-04 Thread Oleg Kalnichevski
On Thu, 2013-07-04 at 12:22 +0100, Ke Ren wrote: > I still get NoHttpResponseException after I turned off keepAlive and stale > check. Do you mean closing connection every time after a request? Absolutely not. In some cases proactive eviction of idle connections from the connection pool makes stal

Re: HttpClient performance question

2013-07-04 Thread Ke Ren
ok, I got it. this stale check is fixed by adding retry if connection is closed by server after response is sent On Thu, Jul 4, 2013 at 12:41 PM, Oleg Kalnichevski wrote: > On Thu, 2013-07-04 at 12:22 +0100, Ke Ren wrote: > > I still get NoHttpResponseException after I turned off keepAlive and