Re: Performance

Oleg, ContentLengthInputStream extend java.io.FilterInputStream, which supports mark and reset. However, ContentLengthInputStream does not keep track of when these are called and appropriately adjust the current posistion. It keeps incrementing the counter on each read() without decrementing it on

Re: Performance

Bjarne, Can you send the a patch (diff -u) against HTTPCLIENT_2_0 branch or just a snippet of the offending code? It will help me avoid manual comparison of two files Oleg On Wed, 2004-09-29 at 21:33, Bjarne Rasmussen wrote: > We've tried out the nightly and think there may be an issue in the >

Re: Performance

We've tried out the nightly and think there may be an issue in the mark/reset code of org.apache.commons.httpclient.ContentLengthInputStream.java (please see attached for rudimentary fix). With this fix performance and functionality is great. Thanks, Bjarne. >>> [EMAIL PROTECTED] 9/28/2004 11:52:

Re: Performance

Sounds good to me. Is there anything else we want to sneak in at the same time? If I don't hear any objections or ideas for other things to include in 2.0.2 I'll call for a vote in the morning. Also, it would be nice to have some specifics about the performance increase to include in the

Re: Performance

Great news indeed. The reported performance boost does justify cutting a new release. Folks, how do you all feel about releasing HttpClient 2.0.2? Oleg On Tue, 2004-09-28 at 00:38, Eric Johnson wrote: > And I've finally gotten test results back from the appropriate people here. > > In our test l

Re: Performance

This is very good news, Eric! Thanks a lot. Eric Johnson wrote: And I've finally gotten test results back from the appropriate people here. In our test lab, between HttpClient 2.0.1 and the nightly, we found a difference of about 4ms per request. As this was a live-test environment, with all of

Re: Performance

And I've finally gotten test results back from the appropriate people here. In our test lab, between HttpClient 2.0.1 and the nightly, we found a difference of about 4ms per request. As this was a live-test environment, with all of our application environment around HttpClient, the total number

Re: Performance

Folks, Could you please grab the latest 2.0 nightly build and see if it runs stable enough for production purposes? When we have a couple of reports confirming adequate stability, we'll call for the 2.0.2 release Oleg On Fri, 2004-09-03 at 00:00, Eric Johnson wrote: > My read on Odi's statistic

Re: Performance

My read on Odi's statistics is that the patch has a pretty consistent 1ms impact on every request. This corresponds pretty well with my understanding of the theoretical improvements behind the patch. To the effect that HttpClient's performance is affected, header parsing will be faster, and r

Re: Performance

Original Message- > >From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] > >Sent: Wednesday, September 01, 2004 2:11 PM > >To: Commons HttpClient Project > >Subject: Re: Performance > > > >Makes sense. I'll start working on a patch. > > > &g

Re: Performance

o change API to take advantageof this. regards Andre -Original Message- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 01, 2004 2:11 PM To: Commons HttpClient Project Subject: Re: Performance Makes sense. I'll start w

RE: Performance

PM > To: Commons HttpClient Project > Subject: Re: Performance > > Makes sense. I'll start working on a patch. > > Oleg > > On Wed, 2004-09-01 at 19:43, Bjarne Rasmussen wrote: > > Hi Oleg, > > > > Forgot to mention: > > > > It&#x

RE: Performance

, September 01, 2004 2:11 PM To: Commons HttpClient Project Subject: Re: Performance Makes sense. I'll start working on a patch. Oleg On Wed, 2004-09-01 at 19:43, Bjarne Rasmussen wrote: > Hi Oleg, > > Forgot to mention: > > It's not that BufferedInputStream#reset

Re: Performance

Makes sense. I'll start working on a patch. Oleg On Wed, 2004-09-01 at 19:43, Bjarne Rasmussen wrote: > Hi Oleg, > > Forgot to mention: > > It's not that BufferedInputStream#reset is better or faster than > PushbackInputStream#unread. The BufferedInputStream merely supports > mark/reset as a s

Re: Performance

Hi Oleg, Forgot to mention: It's not that BufferedInputStream#reset is better or faster than PushbackInputStream#unread. The BufferedInputStream merely supports mark/reset as a side-effect of doing buffering. The PushbackInputStream doesn't buffer. From HttpParser: public static byte[] readR

Re: Performance

Hi Oleg, I ran this by Ortwin prior to posting here: > I ran some quick tests. I just changed the line to: > inputStream = new PushbackInputStream(new > BufferedInputStream(socket.getInputStream())); > and left everything else as it is. > > Running 5000 GET requests to a local Tomcat yields:

Re: Performance

Bjarne, Could you define 'considerably' in some ANSI units? ;-) I have just recently dealt with this problem. The empirical data that I got appears to suggest that there's a (more or less) constant delta in performance of ~2-3ms, which only makes a difference for relatively small payloads. For mo

Re: Performance - ConnectionManager vs none?

> Adrian Sutton wrote: >> I've recently been doing some profiling work and was surprised to discover >> that instantiating HttpClient is actually a reasonably expensive operation. > > Sounds interesting! Do you know what the reason is? Where is all that > time spent? I mean, HttpClient doesn't do

Re: Performance - ConnectionManager vs none?

Adrian Sutton wrote: I've recently been doing some profiling work and was surprised to discover that instantiating HttpClient is actually a reasonably expensive operation. Sounds interesting! Do you know what the reason is? Where is all that time spent? I mean, HttpClient doesn't do that much. I

Re: Performance - ConnectionManager vs none?

> As Odi pointed out, you can save the overhead of establishing > a new connection for each request. How much that is depends > on your infrastructure: network latency, authentication overhead,... I've recently been doing some profiling work and was surprised to discover that instantiating HttpCli

Re: Performance - ConnectionManager vs none?

Roland Weber wrote: As Odi pointed out, you can save the overhead of establishing a new connection for each request. How much that is depends on your infrastructure: network latency, authentication overhead,... Also this means, for a very special applications you can even get more speed or save

Re: Performance - ConnectionManager vs none?

Hello André-John, if you continue to create a new HTTP Client in each thread, there is no point in using the multi threaded connection manager. That makes sense only if several threads share the same HTTP Client. That's more effort than just changing a single line of code. As Odi pointed out, you

Re: Performance - ConnectionManager vs none?

André-John Mas wrote: Now I am wondering how much of performance increase, if any, there would be It would just reuse open connections to issue requests. So the overhead of re-opening connection may be reduced. - To unsubscrib

Re: Performance Issue/JBoss/HttpClient ...

Please let me sumurized ... I have 2 computer. 1 with JBoss know as the server, and 1 with eclipse and JBoss, know as the workstation. Both JBoss are working in clusturing when started. When I start JBoss on the serveur and the Swing application client on the workstation, connecting to the serv

Re: Performance Issue/JBoss/HttpClient ...

About my investigation ... At the line 109 of org.apache.commons.httpclient.HttpParser, i'm waiting 45s for ending of ch = inputStream.read() ... So, it seems it's independent of HttpClient ... But that I don't understand is that the server do not received anything the 40 first seconds. So, why us

Re: Performance Issue/JBoss/HttpClient ...

Jean-Marc, At the moment I tend to think that the problem seems to be on the server-side. See if JBoss' logs can reveal anything useful. Another thing to try is disabling stale connection check SimpleHttpConnectionManager connman = new SimpleHttpConnectionManager(); connman.setConnectionStaleChec

Re: Performance Issue/JBoss/HttpClient ...

Jean-Marc, I am afraid it is not possible to give you a definitive answer unless we know in greater details where exactly the time is being spent. Take a look at the HttpClient logging guide and try to produce the wirelog for HTTP sessions

Re: Performance Issue/JBoss/HttpClient ...

> It seems the first time I call executeMethod, all is working fine. But > only the first time. > Jean-Marc, I am afraid it is not possible to give you a definitive answer unless we know in greater details where exactly the time is being spent. Take a look at the HttpClient logging guide

Re: Performance HTTPClient vs. ab

Wolfgang Hoschek - DSD Staff wrote: - disabling stale connections with connectionMgr.setConnectionStaleCheckingEnabled(false) improves performance by a factor of 50 (note: factor, not percent). This is because stale connection checking adds a constant penalty time, which for very quick requests

RE: Performance HTTPClient vs. ab

Oleg, Thanks for the kind help with working, well-performing code, which resolves the problem according to bench runs here. Just for the record, so that others may benefit as well, bench runs for *this* use case show the following: - GetMethod vs. PostMethod makes no noticeable difference - Simp

RE: Performance HTTPClient vs. ab

Wolfgang, Please allow me to point out that your comparison is not completely accurate. Apache benchmark (I assume) uses HTTP GET requests, whereas your test application uses HTTP POST requests. Besides, you should have used HttpCliet's simple (non-multithreaded) connection manager to be on an

Re: Performance HTTPClient vs. ab

Wolfgang, please try using the SimpleHttpConnectionManager instead of MultiThreadedHttpConnectionManager. Odi Wolfgang Hoschek wrote: I am planning on sending and receiving MANY (tens of tousands) small SOAP-like serial messages over the same persistent HTTP 1.1 connection, using a single cli

Re: performance issue

ttpClient Project" <[EMAIL PROTECTED]> Sent: Tuesday, July 29, 2003 8:37 AM Subject: Re: performance issue > It does appear to JDK 1.4.0. I just installed 1.4.0_04 on Windows 2000 > and beta2 is significantly slower (almost by a factor of 10). Removing > the isStale() check f

Re: performance issue

Take a look at Sun BUG ID 4512028. This bug was introduced in 1.4.0 and was fixed in 1.4.1. It appears to effect only Windows. Mike Michael Becke wrote: It does appear to JDK 1.4.0. I just installed 1.4.0_04 on Windows 2000 and beta2 is significantly slower (almost by a factor of 10). Remov

Re: performance issue

It does appear to JDK 1.4.0. I just installed 1.4.0_04 on Windows 2000 and beta2 is significantly slower (almost by a factor of 10). Removing the isStale() check fixes the problem in this case. It seems 1.4.0 does not do well with SO_TIMEOUT. This does not seem to be a problem with Sun 1.3.

RE: performance issue

3 5:18 PM To: [EMAIL PROTECTED] Subject: RE: performance issue Well ... there must be something different about my environment. The results with JBoss3.2.1-tomcat4.1.24 were much worse! (I decreased threads to 5 and messages per thread to 10 since tomcat defaults max threads per servlet somewhe

Re: Performance Issue

PROTECTED]> Sent: Tuesday, July 29, 2003 5:55 AM Subject: RE: Performance Issue Eric, All credits should go to Todd. He's done pretty much all the hard work. Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Base

RE: performance issue

Well ... there must be something different about my environment. The results with JBoss3.2.1-tomcat4.1.24 were much worse! (I decreased threads to 5 and messages per thread to 10 since tomcat defaults max threads per servlet somewhere and I didn't want to dig for the setting.) alpha3 .034 minute

Re: Performance Issue

Todd Wolff wrote: Mike, that certainly makes sense. My intent was to eliminate any wait time for a connection and not to have to pool the connections myself - nor have to deal with any extraneous connections issues, ie. isStale ;-) I hear you. It is certainly okay to do it this way. I just wante

RE: Performance Issue

> As far as disabling isStale(), where do we want to add the preference > for this? Is it per HttpConnectionManager or HttpClient? I'd say per HttpConnectionManager. Disabling 'state' connection check should be considered fine-tuning. Very few folks would really need to do so. There's no point

Re: Performance Issue

> Todd, one thing I noticed is that you are setting maxConnectionsPerHost > to the number of threads you are creating. This pretty much defeats > the purpose of using the MTCM and there is no need to share > connections. There are enough connections for each thread to just own > one. Mike, t

RE: Performance Issue

mmons HttpClient Project Subject: Re: Performance Issue Sure, the code is yours to do with as you wish. Related to the performance issue, I am still seeing a degradation by a factor of roughly 2.5. - and yes, logging is set to error level :-) The problem may be isolated to httpclient's intera

Re: Performance Issue

From: "Kalnichevski, Oleg" <[EMAIL PROTECTED]> To: "Commons HttpClient Project" <[EMAIL PROTECTED]> Sent: Tuesday, July 29, 2003 5:55 AM Subject: RE: Performance Issue Eric, All credits should go to Todd. He's done pretty much all the hard work. > Maybe it is ju

Re: Performance Issue

Kalnichevski, Oleg wrote: All credits should go to Todd. He's done pretty much all the hard work. Agreed. Thank you Todd for bringing this up and for the test code. Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Based

Re: Performance Issue

Eric Johnson wrote: Ortwin, So far as I know, since Windows 95 it has been possible to measure and wait for time intervals of 1ms. With Windows NT4.0/2000 and later, it is even possible to do microseconds. I think the _really_ coarse timers are a hang-over from Windows 3.1 days, and perhaps s

Re: Performance Issue

Ortwin, So far as I know, since Windows 95 it has been possible to measure and wait for time intervals of 1ms. With Windows NT4.0/2000 and later, it is even possible to do microseconds. I think the _really_ coarse timers are a hang-over from Windows 3.1 days, and perhaps some of the really e

RE: Performance Issue

Eric Johnson a écrit : > > Maybe it is just me, but I can live with a 1ms penalty that > dramatically increases the reliability of the re-used connections. > Based on your research, I think we should keep the isStale() check. > What do others think? Totally agree, I was getting mad before the isS

RE: Performance Issue

Eric, All credits should go to Todd. He's done pretty much all the hard work. > Maybe it is just me, but I can live with a 1ms penalty that dramatically > increases the reliability of the re-used connections. Based on your > research, I think we should keep the isStale() check. What do others

Re: Performance Issue

Eric Johnson wrote: since isStale() calls setSoTimeout(1), since it cannot set it to zero. Is this platform safe actually? Windows has only very corse timers, will it still be around 1ms or will it be 20-30ms? Maybe it is just me, but I can live with a 1ms penalty that dramatically increases t

Re: Performance Issue

Oleg, Thanks for doing the research! If my math is correct, this means: 2.0b2 - average of 10.23ms per request 2.0rc1 without isStale - average of 8.16ms per request This would seem to correlate exactly with the code - we know that the penalty of calling isStale() should be approximately one mil

RE: Performance Issue

OK. I did a stupid thing that flawed all my measurements. I have basically ended up measuring the speed of console output ;-). Dumb Russian I am. Todd, are you sure you have not fallen into the same trap? BETA2 is simply more verbose than ALPHA3 ;-) There are the revised numbers (with wire log

RE: Performance Issue

mons HttpClient Project Subject: Re: Performance Issue Just sending a quick note before I go to bed. I did some testing with the example code and am now quite confident that HTCM is doing what it should. I have not had time to add any DEBUG code but hopefully I will get to it tomorrow. Todd, one

Re: Performance Issue

Oleg Kalnichevski wrote: A clear drawback to this approach is that the connection would need to 'know' what kind of request is being executed. Of course, this is just design purism. Feel free to ignore me. Oleg Uhm... the connection does not need to know the method by name. It's sufficient that

Re: Performance Issue

ou and Mike directly rather than taking up bandwidth on the list (it's 250k.) Todd - Original Message - From: "Oleg Kalnichevski" <[EMAIL PROTECTED]> To: "Commons HttpClient Project" <[EMAIL PROTECTED]> Sent: Sunday, July 27, 2003 7:10 AM Subject: Re:

Re: Performance Issue

Michael Becke wrote: > Oleg Kalnichevski wrote: > * Perhaps the "isStale" check is unnecessary for methods that can simply repeat themselves, for example, GET, HEAD, OPTION, TRACE. For those methods, we could allow a "retry" to fix the problem. For methods such as

Re: Performance Issue

This is a possibility I think. As Oleg mentioned I don't think we want the connection to know about the method. We should be able to accomplish something like this in the HttpMethod though. This assumes that we can turn on/off the isStale test from outside the connection. Mike Oleg Kalniche

Re: Performance Issue

> >* Perhaps the "isStale" check is unnecessary for methods that can > > simply repeat themselves, for example, GET, HEAD, OPTION, TRACE. > > For those methods, we could allow a "retry" to fix the problem. > > For methods such as POST and PUT, however, the isStale is probably

Re: Performance Issue

ng up bandwidth on the list > (it's 250k.) > > > > Todd > > > - Original Message - > From: "Oleg Kalnichevski" <[EMAIL PROTECTED]> > To: "Commons HttpClient Project" <[EMAIL PROTECTED]> > Sent: Sunday, July 27, 2003 7:10

Re: Performance Issue

Eric Johnson wrote: Some possible alternatives: * Only do the isStale check if the connection has been sitting in the pool for a configurable amount of time. I'm guessing we could choose a value here between 5 and 30 seconds without any significant change in behavior, that is to

Re: Performance Issue

Kalnichevski, Oleg wrote: I would just make 'stale' connection check optional and off per default. Making the check pluggable would require too much of a change for what is essentially a release candidate. That should be better done as a part of 3.0 redesign IMO. As I recall, the "isStale" functi

Re: Performance Issue

andwidth on the list (it's 250k.) Todd - Original Message - From: "Oleg Kalnichevski" <[EMAIL PROTECTED]> To: "Commons HttpClient Project" <[EMAIL PROTECTED]> Sent: Sunday, July 27, 2003 7:10 AM Subject: Re: Performance Issue > Todd, > >

RE: Performance Issue

> I agree that removing isStale() by default certainly fixes the > performance problem. However, it makes MTCM somewhat unstable. The > case where stale connections do not cause errors until read from will > throw unrecoverable exceptions. Is this an acceptable alternative? What if we keep '

Re: Performance Issue

On Monday, July 28, 2003, at 06:10 AM, Kalnichevski, Oleg wrote: I would just make 'stale' connection check optional and off per default. Making the check pluggable would require too much of a change for what is essentially a release candidate. That should be better done as a part of 3.0 redesi

RE: Performance Issue

> I believe our only real option is to make isStale() optional and/or > pluggable. Effective alternatives to isStale() have been thoroughly > investigated and I think we can say that they do not exist, at least in > a pre 1.4 environment. Does anyone have some new insight into this > problem?

Re: Performance Issue

On Sunday, July 27, 2003, at 12:39 PM, Oleg Kalnichevski wrote: I suppose this particular problem is not related to isStale(), as I do not see in the logs any connections dropped because they appear to be 'stale'. I still lean toward assuming this is an issue with Todd' code, rather HttpClient itse

Re: Performance Issue

> - Do the recently added DEBUG messages in the > MultiThreadedHttpConnectionManager help at all? Additional DEBUG messages did not make it into beta-2, which Todd has been using. We may ask him to upgrade to what has been tagged as HTTPCLIENT_2_0_RC1. > - Are we sure the the connection man

Re: Performance Issue

Oleg, Todd, I just wanted to let you know that I am listening. I will not be able to work on this for a few hours, but I should have plenty of time this evening. I have not looked into this closely but I have a few initial questions: - Do the recently added DEBUG messages in the MultiThread

Re: Performance Issue

Todd, I think I have identified the problem or one of the problem that contributed to the noticeable performance degradation for short HTTP messages in post 2.0a3 releases. So, many thanks for bringing up this issue Further, see my comments below On Sun, 2003-07-27 at 03:36, Todd Wolff wrote

Re: Performance Issue

ed per request. Is there anything I can do to encourage re-use? Gracias, Todd - Original Message - From: "Oleg Kalnichevski" <[EMAIL PROTECTED]> To: "Commons HttpClient Project" <[EMAIL PROTECTED]> Sent: Saturday, July 26, 2003 1:48 AM Subject: Re: Per

Re: Performance Issue

I think I tell the reason even without having seen the code. Since beta-1 'expect: 100-continue' handshake is off per default, that can make a huge difference in terms of performance with POST requests that require authentication. Just do the following httppost.setUseExpectHeader(true); http://

Re: performance

Zulfi Umrani and Ortwin Glück wrote: I will appreciate if anyone answers my questions below. -Does anyone has perfromance comparison between HTTPClient and JDK HttpURLConnection? I have done some comparisons. We do some concurrent requests to a tomcat servlet to fill JTables. The communication

Re: performance

Zulfi Umrani wrote: I will appreciate if anyone answers my questions below. -Does anyone has perfromance comparison between HTTPClient and JDK HttpURLConnection? I remember there was a comment about that on the list some months ago. The guy said that HttpClient was slightly faster than HttpUrlCo

RE: performance

Derk Muenchhausen wrote : > Was my measuring wrong? Is HttpClient really faster than > HttpURLConnection? What have you done, that HttpClient got significant > faster? > > Derk In my case, my first attempt for my project was also using jdk only, but I had to improve response performance/delay. I

Re: performance

Yes I guess it's the TCP_NODELAY (enabled by default?). Sung-Gu wrote: I guess TCP is set by no delay to send packets... - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Re: performance

There have been a few performance enhancements along the way, but we have never gone through a performance optimizing phase. Sometime during late beta that should be done. Its encouraging that you are finding it fast now though! Kalnichevski, Oleg wrote: I am not aware of HttpClient being s

Re: performance

I guess TCP is set by no delay to send packets... Sung-Gu - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

RE: performance

I am not aware of HttpClient being specifically optimized for performance. May other developers correct me if I err, rather, I believe, HttpClient is being developed primarily with standards compliance in mind. However, I am by no means entitled to speak on behalf of all HttpClient developers, s