Re: Performance Issue/JBoss/HttpClient ...
It seems the first time I call executeMethod, all is working fine. But only the first time. Jean-Marc, I am afraid it is not possible to give you a definitive answer unless we know in greater details where exactly the time is being spent. Take a look at the HttpClient logging guide http://jakarta.apache.org/commons/httpclient/logging.html and try to produce the wirelog for HTTP sessions with and without the said performance problem. The wirelogs output comparison may reveal the difference between these two session and may provide hints as to what the cause of the problem might be or at the very least where the execution time is being spent. Cheers, Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue/JBoss/HttpClient ...
Jean-Marc, I am afraid it is not possible to give you a definitive answer unless we know in greater details where exactly the time is being spent. Take a look at the HttpClient logging guide http://jakarta.apache.org/commons/httpclient/logging.html and try to produce the wirelog for HTTP sessions with and without the said performance problem. The wirelogs output comparison may reveal the difference between these two session and may provide hints as to what the cause of the problem might be or at the very least where the execution time is being spent. Cheers, Oleg Hi Oleg, Hi all. Here are the log without JBoss running on my own computer. See below for log with JBoss, and below again for extract where the problem occurs. 2004/01/25 17:10:10:484 EST [DEBUG] HttpConnection - -Connection is stale, closing... 2004/01/25 17:10:10:486 EST [DEBUG] HttpConnection - -HttpConnection.setSoTimeout(0) 2004/01/25 17:10:10:489 EST [DEBUG] HttpMethodBase - -Execute loop try 1 2004/01/25 17:10:10:494 EST [DEBUG] wire - - POST /JTrad/FrontServlet HTTP/1.1[\r][\n] 2004/01/25 17:10:10:494 EST [DEBUG] HttpMethodBase - -Adding Host request header 2004/01/25 17:10:10:496 EST [DEBUG] wire - - User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6b) Gecko/20031210[\r][\n] 2004/01/25 17:10:10:496 EST [DEBUG] wire - - Host: maison.spaggiari.org:8180[\r][\n] 2004/01/25 17:10:10:497 EST [DEBUG] wire - - Cookie: $Version=0; JSESSIONID=IeC5xv-Yw6mpH5WnyKRmWg**; $Path=/JTrad[\r][\n] 2004/01/25 17:10:10:498 EST [DEBUG] wire - - Content-Length: 128[\r][\n] 2004/01/25 17:10:10:498 EST [DEBUG] wire - - [\r][\n] 2004/01/25 17:10:10:499 EST [DEBUG] EntityEnclosingMethod - -Using buffered request body 2004/01/25 17:10:10:499 EST [DEBUG] wire - - [0xfffd][0xfffd][0x0][0x5]sr[0x0][0x13]java.util.Hashtable[0x13][0xfffd][0xf]%!J[0xfffd][0xfffd][0x3][0x0][0x2]F[0x0][\n] 2004/01/25 17:10:10:504 EST [DEBUG] wire - - [EMAIL PROTECTED] 2004/01/25 17:10:10:504 EST [DEBUG] EntityEnclosingMethod - -Request body sent 2004/01/25 17:10:10:540 EST [DEBUG] wire - - HTTP/1.1 200 OK[\r][\n] 2004/01/25 17:10:10:543 EST [DEBUG] wire - - Transfer-Encoding: chunked[\r][\n] 2004/01/25 17:10:10:544 EST [DEBUG] wire - - Date: Sun, 25 Jan 2004 22:09:14 GMT[\r][\n] 2004/01/25 17:10:10:544 EST [DEBUG] wire - - Server: Apache-Coyote/1.1[\r][\n] 2004/01/25 17:10:10:545 EST [DEBUG] HttpConnection - -HttpConnection.getSoTimeout() 2004/01/25 17:10:10:720 EST [DEBUG] wire - - 1 2004/01/25 17:10:10:720 EST [DEBUG] wire - - 4 2004/01/25 17:10:10:721 EST [DEBUG] wire - - 8 2004/01/25 17:10:10:721 EST [DEBUG] wire - - [\r] 2004/01/25 17:10:10:722 EST [DEBUG] wire - - [\n] 2004/01/25 17:10:10:723 EST [DEBUG] wire - - [0xfffd][0xfffd] 2004/01/25 17:10:10:723 EST [DEBUG] wire - - [0x0][0x5] 2004/01/25 17:10:10:725 EST [DEBUG] wire - - s 2004/01/25 17:10:10:725 EST [DEBUG] wire - - r 2004/01/25 17:10:10:725 EST [DEBUG] wire - - [0x0][0x10] 2004/01/25 17:10:10:725 EST [DEBUG] wire - - java.util.Vector 2004/01/25 17:10:10:726 EST [DEBUG] wire - - [0xfffd][0xfffd]}[[0xfffd];[0xfffd][0x1] 2004/01/25 17:10:10:726 EST [DEBUG] wire - - [0x3] 2004/01/25 17:10:10:726 EST [DEBUG] wire - - [0x0][0x3] 2004/01/25 17:10:10:726 EST [DEBUG] wire - - I 2004/01/25 17:10:10:727 EST [DEBUG] wire - - [0x0][0x11] 2004/01/25 17:10:10:727 EST [DEBUG] wire - - capacityIncrement 2004/01/25 17:10:10:727 EST [DEBUG] wire - - I 2004/01/25 17:10:10:730 EST [DEBUG] wire - - [0x0][0xc] 2004/01/25 17:10:10:731 EST [DEBUG] wire - - elementCount 2004/01/25 17:10:10:732 EST [DEBUG] wire - - [ 2004/01/25 17:10:10:732 EST [DEBUG] wire - - [0x0][0xb] 2004/01/25 17:10:10:732 EST [DEBUG] wire - - elementData 2004/01/25 17:10:10:733 EST [DEBUG] wire - - t 2004/01/25 17:10:10:733 EST [DEBUG] wire - - [0x0][0x13] 2004/01/25 17:10:10:734 EST [DEBUG] wire - - [Ljava/lang/Object; 2004/01/25 17:10:10:734 EST [DEBUG] wire - - x 2004/01/25 17:10:10:735 EST [DEBUG] wire - - p 2004/01/25 17:10:10:745 EST [DEBUG] wire - - [0x0][0x0][0x0][0x0][0x0][0x0][0x0][0x3] 2004/01/25 17:10:10:746 EST [DEBUG] wire - - u 2004/01/25 17:10:10:747 EST [DEBUG] wire - - r 2004/01/25 17:10:10:747 EST [DEBUG] wire - - [0x0][0x13] 2004/01/25 17:10:10:747 EST [DEBUG] wire - - [Ljava.lang.Object; 2004/01/25 17:10:10:748 EST [DEBUG] wire - - [0xfffd][0xfffd]X[0xfffd][0x10]s)l 2004/01/25 17:10:10:748 EST [DEBUG] wire - - [0x2] 2004/01/25 17:10:10:749 EST [DEBUG] wire - - [0x0][0x0] 2004/01/25 17:10:10:749 EST [DEBUG] wire - - x 2004/01/25 17:10:10:749 EST [DEBUG] wire - - p 2004/01/25 17:10:10:756 EST [DEBUG] wire - - [0x0][0x0][0x0][\n] 2004/01/25 17:10:10:757 EST [DEBUG] wire - - s 2004/01/25 17:10:10:757 EST [DEBUG] wire - - r 2004/01/25 17:10:10:758 EST [DEBUG] wire - - [0x0]# 2004/01/25 17:10:10:758 EST [DEBUG] wire - - org.spaggiari.jtrad.donnees.Domaine 2004/01/25 17:10:10:759 EST [DEBUG] wire - - [0xfffd]=[0xfffd]1[0x1c][0xfffd][0x10][0xfffd]
Re: Performance Issue/JBoss/HttpClient ...
Jean-Marc, At the moment I tend to think that the problem seems to be on the server-side. See if JBoss' logs can reveal anything useful. Another thing to try is disabling stale connection check SimpleHttpConnectionManager connman = new SimpleHttpConnectionManager(); connman.setConnectionStaleCheckingEnabled(true); HttpClient client = new HttpClient(connman); http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpConnection.html#setStaleCheckingEnabled(boolean) See if that makes any difference in terms of performance Oleg On Sun, 2004-01-25 at 23:21, Jean-Marc Spaggiari wrote: Jean-Marc, I am afraid it is not possible to give you a definitive answer unless we know in greater details where exactly the time is being spent. Take a look at the HttpClient logging guide http://jakarta.apache.org/commons/httpclient/logging.html and try to produce the wirelog for HTTP sessions with and without the said performance problem. The wirelogs output comparison may reveal the difference between these two session and may provide hints as to what the cause of the problem might be or at the very least where the execution time is being spent. Cheers, Oleg Hi Oleg, Hi all. Here are the log without JBoss running on my own computer. See below for log with JBoss, and below again for extract where the problem occurs. 2004/01/25 17:10:10:484 EST [DEBUG] HttpConnection - -Connection is stale, closing... 2004/01/25 17:10:10:486 EST [DEBUG] HttpConnection - -HttpConnection.setSoTimeout(0) 2004/01/25 17:10:10:489 EST [DEBUG] HttpMethodBase - -Execute loop try 1 2004/01/25 17:10:10:494 EST [DEBUG] wire - - POST /JTrad/FrontServlet HTTP/1.1[\r][\n] 2004/01/25 17:10:10:494 EST [DEBUG] HttpMethodBase - -Adding Host request header 2004/01/25 17:10:10:496 EST [DEBUG] wire - - User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6b) Gecko/20031210[\r][\n] 2004/01/25 17:10:10:496 EST [DEBUG] wire - - Host: maison.spaggiari.org:8180[\r][\n] 2004/01/25 17:10:10:497 EST [DEBUG] wire - - Cookie: $Version=0; JSESSIONID=IeC5xv-Yw6mpH5WnyKRmWg**; $Path=/JTrad[\r][\n] 2004/01/25 17:10:10:498 EST [DEBUG] wire - - Content-Length: 128[\r][\n] 2004/01/25 17:10:10:498 EST [DEBUG] wire - - [\r][\n] 2004/01/25 17:10:10:499 EST [DEBUG] EntityEnclosingMethod - -Using buffered request body 2004/01/25 17:10:10:499 EST [DEBUG] wire - - [0xfffd][0xfffd][0x0][0x5]sr[0x0][0x13]java.util.Hashtable[0x13][0xfffd][0xf]%!J[0xfffd][0xfffd][0x3][0x0][0x2]F[0x0][\n] 2004/01/25 17:10:10:504 EST [DEBUG] wire - - [EMAIL PROTECTED] 2004/01/25 17:10:10:504 EST [DEBUG] EntityEnclosingMethod - -Request body sent 2004/01/25 17:10:10:540 EST [DEBUG] wire - - HTTP/1.1 200 OK[\r][\n] 2004/01/25 17:10:10:543 EST [DEBUG] wire - - Transfer-Encoding: chunked[\r][\n] 2004/01/25 17:10:10:544 EST [DEBUG] wire - - Date: Sun, 25 Jan 2004 22:09:14 GMT[\r][\n] 2004/01/25 17:10:10:544 EST [DEBUG] wire - - Server: Apache-Coyote/1.1[\r][\n] 2004/01/25 17:10:10:545 EST [DEBUG] HttpConnection - -HttpConnection.getSoTimeout() 2004/01/25 17:10:10:720 EST [DEBUG] wire - - 1 2004/01/25 17:10:10:720 EST [DEBUG] wire - - 4 2004/01/25 17:10:10:721 EST [DEBUG] wire - - 8 2004/01/25 17:10:10:721 EST [DEBUG] wire - - [\r] 2004/01/25 17:10:10:722 EST [DEBUG] wire - - [\n] 2004/01/25 17:10:10:723 EST [DEBUG] wire - - [0xfffd][0xfffd] 2004/01/25 17:10:10:723 EST [DEBUG] wire - - [0x0][0x5] 2004/01/25 17:10:10:725 EST [DEBUG] wire - - s 2004/01/25 17:10:10:725 EST [DEBUG] wire - - r 2004/01/25 17:10:10:725 EST [DEBUG] wire - - [0x0][0x10] 2004/01/25 17:10:10:725 EST [DEBUG] wire - - java.util.Vector 2004/01/25 17:10:10:726 EST [DEBUG] wire - - [0xfffd][0xfffd]}[[0xfffd];[0xfffd][0x1] 2004/01/25 17:10:10:726 EST [DEBUG] wire - - [0x3] 2004/01/25 17:10:10:726 EST [DEBUG] wire - - [0x0][0x3] 2004/01/25 17:10:10:726 EST [DEBUG] wire - - I 2004/01/25 17:10:10:727 EST [DEBUG] wire - - [0x0][0x11] 2004/01/25 17:10:10:727 EST [DEBUG] wire - - capacityIncrement 2004/01/25 17:10:10:727 EST [DEBUG] wire - - I 2004/01/25 17:10:10:730 EST [DEBUG] wire - - [0x0][0xc] 2004/01/25 17:10:10:731 EST [DEBUG] wire - - elementCount 2004/01/25 17:10:10:732 EST [DEBUG] wire - - [ 2004/01/25 17:10:10:732 EST [DEBUG] wire - - [0x0][0xb] 2004/01/25 17:10:10:732 EST [DEBUG] wire - - elementData 2004/01/25 17:10:10:733 EST [DEBUG] wire - - t 2004/01/25 17:10:10:733 EST [DEBUG] wire - - [0x0][0x13] 2004/01/25 17:10:10:734 EST [DEBUG] wire - - [Ljava/lang/Object; 2004/01/25 17:10:10:734 EST [DEBUG] wire - - x 2004/01/25 17:10:10:735 EST [DEBUG] wire - - p 2004/01/25 17:10:10:745 EST [DEBUG] wire - - [0x0][0x0][0x0][0x0][0x0][0x0][0x0][0x3] 2004/01/25 17:10:10:746 EST [DEBUG] wire - - u 2004/01/25 17:10:10:747 EST [DEBUG] wire - - r 2004/01/25 17:10:10:747 EST [DEBUG] wire - - [0x0][0x13] 2004/01/25
Re: Performance Issue/JBoss/HttpClient ...
About my investigation ... At the line 109 of org.apache.commons.httpclient.HttpParser, i'm waiting 45s for ending of ch = inputStream.read() ... So, it seems it's independent of HttpClient ... But that I don't understand is that the server do not received anything the 40 first seconds. So, why us HttpClient or my computer waiting 40s before sending request only when Jboss is active ? I don't know. Is the send metho of HttpClient multi threaded ? I will look now in this direction, by at this time, I have no idea where is the probleme :( It's not from log metho because isDebugEnabled return false. I will continue and give you some news soon (I hope). JMS. Make sure you use HTTP_CLIENT_2_0_BRANCH tag when checking the source code out. CVS HEAD contains development version (pre 3.0-alpha) which is no longer API compatible with 2.0 Keep us posted on the progress of your investigation Oleg On Mon, 2004-01-26 at 00:06, [EMAIL PROTECTED] wrote: Client side : 2004/01/25 17:40:21:346 EST [DEBUG] EntityEnclosingMethod - -Request body sent 2004/01/25 17:41:20:467 EST [DEBUG] wire - - HTTP/1.1 200 OK[\r][\n] Server side : 17:41:19,113 INFO [STDOUT] Here ! I have synchro my both machine with ntpdate ... I have try with setConnectionStaleCheckingEnabled(true); and setConnectionStaleCheckingEnabled(false); but it's not working any more. Here are my log levels (int $JBOSSHOME/server/all/conf/log4j.xml) : category name=org.apache.commons priority value=ERROR/ /category category name=org.apache.jcs priority value=INFO/ /category !-- Limit JBoss categories to INFO -- category name=org.jboss priority value=INFO/ /category And all my log files was empty. I think I will take the src files of HttpClient from CVS and try to trace into what's wrong with my configuration :( JMS. Jean-Marc, At the moment I tend to think that the problem seems to be on the server-side. See if JBoss' logs can reveal anything useful. Another thing to try is disabling stale connection check SimpleHttpConnectionManager connman = new SimpleHttpConnectionManager(); connman.setConnectionStaleCheckingEnabled(true); HttpClient client = new HttpClient(connman); http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpConnection.html#setStaleCheckingEnabled(boolean) See if that makes any difference in terms of performance Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue/JBoss/HttpClient ...
Please let me sumurized ...
I have 2 computer.
1 with JBoss know as the server, and 1 with eclipse and JBoss, know as the
workstation.
Both JBoss are working in clusturing when started.
When I start JBoss on the serveur and the Swing application client on the
workstation, connecting to the serveur, is working fine.
When I start JBoss on the workstation and connecting the Swing application
on workstation too, it's working fine.
When I start JBoss on workstation and on server, and I connect the swing
application on the workstation, it's working fine.
When I start JBoss on workstation and on server, and I connect the swing
applicatino on the serveur, it's not working :(
In the client application, I'm calling many time the same object for
transfert data. The first time, it's working fine. But the second time,
it'm waitin 1m for the response. I have look on the network with tcpdump,
and the workstation is sending information to server only after a 45s
laps. So, it's not comming from the server.
What I don't understant is why it is working fine for the first connexion,
but not after. Maybe I have miss something ?
Here is my code for sending information. Did you see wrong path ?
{
ByteArrayOutputStream os = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(os);
Hashtable table = new Hashtable ();
table.put(HANDLER, handler);
table.put(SOUSHANDLER, sousHanler);
if (donnees != null)
table.put(DONNEES, donnees);
oos.writeObject(table);
oos.flush();
PostMethod post = null;
post = new PostMethod(http://192.168.23.1:8180/FrontServlet;);
post.setRequestHeader(new Header(User-Agent, Mozilla/5.0 (X11; U; Linux
i686; en-US; rv:1.6b) Gecko/20031210));
post.setRequestBody(new ByteArrayInputStream(os.toByteArray()));
oos.close ();
os.close ();
client.executeMethod(post);
is = post.getResponseBodyAsStream();
actif = true;
}
And a thread is reading is. When the stream is finish, I don't do any
particular things ... Need I ?
JMS.
About my investigation ...
At the line 109 of org.apache.commons.httpclient.HttpParser, i'm waiting
45s for ending of ch = inputStream.read() ... So, it seems it's
independent of HttpClient ... But that I don't understand is that the
server do not received anything the 40 first seconds. So, why us
HttpClient or my computer waiting 40s before sending request only when
Jboss is active ? I don't know. Is the send metho of HttpClient multi
threaded ? I will look now in this direction, by at this time, I have no
idea where is the probleme :(
It's not from log metho because isDebugEnabled return false.
I will continue and give you some news soon (I hope).
JMS.
Make sure you use HTTP_CLIENT_2_0_BRANCH tag when checking the source
code out. CVS HEAD contains development version (pre 3.0-alpha) which is
no longer API compatible with 2.0
Keep us posted on the progress of your investigation
Oleg
On Mon, 2004-01-26 at 00:06, [EMAIL PROTECTED] wrote:
Client side :
2004/01/25 17:40:21:346 EST [DEBUG] EntityEnclosingMethod - -Request
body
sent
2004/01/25 17:41:20:467 EST [DEBUG] wire - - HTTP/1.1 200
OK[\r][\n]
Server side :
17:41:19,113 INFO [STDOUT] Here !
I have synchro my both machine with ntpdate ...
I have try with setConnectionStaleCheckingEnabled(true); and
setConnectionStaleCheckingEnabled(false); but it's not working any
more.
Here are my log levels (int $JBOSSHOME/server/all/conf/log4j.xml) :
category name=org.apache.commons
priority value=ERROR/
/category
category name=org.apache.jcs
priority value=INFO/
/category
!-- Limit JBoss categories to INFO --
category name=org.jboss
priority value=INFO/
/category
And all my log files was empty.
I think I will take the src files of HttpClient from CVS and try to
trace
into what's wrong with my configuration :(
JMS.
Jean-Marc,
At the moment I tend to think that the problem seems to be on the
server-side. See if JBoss' logs can reveal anything useful.
Another thing to try is disabling stale connection check
SimpleHttpConnectionManager connman = new
SimpleHttpConnectionManager();
connman.setConnectionStaleCheckingEnabled(true);
HttpClient client = new HttpClient(connman);
http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpConnection.html#setStaleCheckingEnabled(boolean)
See if that makes any difference in terms of performance
Oleg
-
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Oleg Kalnichevski wrote: A clear drawback to this approach is that the connection would need to 'know' what kind of request is being executed. Of course, this is just design purism. Feel free to ignore me. Oleg Uhm... the connection does not need to know the method by name. It's sufficient that the method control the type of connection it needs. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Performance Issue
OK. I did a stupid thing that flawed all my measurements. I have basically ended up measuring the speed of console output ;-). Dumb Russian I am. Todd, are you sure you have not fallen into the same trap? BETA2 is simply more verbose than ALPHA3 ;-) There are the revised numbers (with wire log disabled, 10 threads, 50 requests per thread) 2.0a3: 500 requests in 0.225 min 2.0b2: 500 requests in 0.086 min 2.0rc1 (with 'stale' connection check removed): 500 requests in 0.068 min This is closer to what I expected to see. I have always suspected that the request buffering should have made beta-versions is fact faster, not slower compared to alpha ones. 'isStale' connection check does slow things down a bit. Is anyone getting different results? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Oleg, Thanks for doing the research! If my math is correct, this means: 2.0b2 - average of 10.23ms per request 2.0rc1 without isStale - average of 8.16ms per request This would seem to correlate exactly with the code - we know that the penalty of calling isStale() should be approximately one millisecond, since isStale() calls setSoTimeout(1), since it cannot set it to zero. Oddly, this doesn't correspond to the 100ms vs 300ms discrepancy reported in the original post of this thread. I wonder if you're correct about the logging overhead problem. Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Based on your research, I think we should keep the isStale() check. What do others think? You might consider committing your performance test as something under the contrib package so that we could look at running it with each release, and thus keep track of the library's performance over time. -Eric. Kalnichevski, Oleg wrote: OK. I did a stupid thing that flawed all my measurements. I have basically ended up measuring the speed of console output ;-). Dumb Russian I am. Todd, are you sure you have not fallen into the same trap? BETA2 is simply more verbose than ALPHA3 ;-) There are the revised numbers (with wire log disabled, 10 threads, 50 requests per thread) 2.0a3: 500 requests in 0.225 min 2.0b2: 500 requests in 0.086 min 2.0rc1 (with 'stale' connection check removed): 500 requests in 0.068 min This is closer to what I expected to see. I have always suspected that the request buffering should have made beta-versions is fact faster, not slower compared to alpha ones. 'isStale' connection check does slow things down a bit. Is anyone getting different results? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Eric Johnson wrote: since isStale() calls setSoTimeout(1), since it cannot set it to zero. Is this platform safe actually? Windows has only very corse timers, will it still be around 1ms or will it be 20-30ms? Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. So can I, however for real-time people 1ms equals eternity... - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Performance Issue
Eric, All credits should go to Todd. He's done pretty much all the hard work. Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Based on your research, I think we should keep the isStale() check. What do others think? I agree. I think this performance penalty is well justified. I would still like to have an option of disabling the check (especially for the single-threaded connection manager), though, as the 'stale' connection check also has some nasty side-effects on SSL connections when running on older JDKs. You might consider committing your performance test as something under the contrib package so that we could look at running it with each release, and thus keep track of the library's performance over time. Todd, would you mind contributing your code to the project? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Performance Issue
Eric Johnson a écrit : Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Based on your research, I think we should keep the isStale() check. What do others think? Totally agree, I was getting mad before the isStale method came up, and I've been moving from b1 to b2 without any trouble here, httpclient works like a charm and is clearly fast. Right now I know I can upgrade my httpclient version without any surprise, and I simply love that ! Aurelien - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Eric Johnson wrote: Ortwin, So far as I know, since Windows 95 it has been possible to measure and wait for time intervals of 1ms. With Windows NT4.0/2000 and later, it is even possible to do microseconds. I think the _really_ coarse timers are a hang-over from Windows 3.1 days, and perhaps some of the really early hardware supported by Windows 95. That's just what I recall, though. -Eric. Ok. I just thought of the bad resolution of System.currentTimeMillis(). Well I guess this has to do with the PC's RTC. So, no worries. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Kalnichevski, Oleg wrote: All credits should go to Todd. He's done pretty much all the hard work. Agreed. Thank you Todd for bringing this up and for the test code. Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Based on your research, I think we should keep the isStale() check. What do others think? I agree. I think this performance penalty is well justified. I would still like to have an option of disabling the check (especially for the single-threaded connection manager), though, as the 'stale' connection check also has some nasty side-effects on SSL connections when running on older JDKs. I have done some more tests this morning as well and I agree with your findings. 2.0 with or without isStale is definitely better than alpha-3 (at least in this case). Especially when using Expect: 100-continue. I believe this functionality was broken in Alpha-3. I also performed tests between RC1 reusing connections with isStale and RC1 without reusing connections (I forced a close each time). The good news here is that reusing connections (even with isStale) is still better than creating a new connection each time. I agree that the small penalty for isStale is worth the added benefit. Most likely there will be some variance depending on platform and JRE but I still think it's worthwhile. As far as disabling isStale(), where do we want to add the preference for this? Is it per HttpConnectionManager or HttpClient? Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Performance Issue
Todd That's correct. I have been using Tomcat 4.1.24 for my tests. Just for the heck of it, do you mind trying to run your test against Tomcat 4.1.x servlet container? Oleg -Original Message- From: Todd Wolff [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 29, 2003 6:36 PM To: Commons HttpClient Project Subject: Re: Performance Issue Sure, the code is yours to do with as you wish. Related to the performance issue, I am still seeing a degradation by a factor of roughly 2.5. - and yes, logging is set to error level :-) The problem may be isolated to httpclient's interaction with the Jetty container. Oleg, I recall you saying that you were using the Tomcat distribution. My results using 10 threads: JBoss-3.0.4(Jetty) alpha350 messages in .036 minutes beta2 50 messages in .084 JBoss - 3.2.1(Jetty) alpha350 messages in .035 minutes beta2 50 messages in .083 - Original Message - From: Kalnichevski, Oleg [EMAIL PROTECTED] To: Commons HttpClient Project [EMAIL PROTECTED] Sent: Tuesday, July 29, 2003 5:55 AM Subject: RE: Performance Issue Eric, All credits should go to Todd. He's done pretty much all the hard work. Maybe it is just me, but I can live with a 1ms penalty that dramatically increases the reliability of the re-used connections. Based on your research, I think we should keep the isStale() check. What do others think? I agree. I think this performance penalty is well justified. I would still like to have an option of disabling the check (especially for the single-threaded connection manager), though, as the 'stale' connection check also has some nasty side-effects on SSL connections when running on older JDKs. You might consider committing your performance test as something under the contrib package so that we could look at running it with each release, and thus keep track of the library's performance over time. Todd, would you mind contributing your code to the project? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Todd, one thing I noticed is that you are setting maxConnectionsPerHost to the number of threads you are creating. This pretty much defeats the purpose of using the MTCM and there is no need to share connections. There are enough connections for each thread to just own one. Mike, that certainly makes sense. My intent was to eliminate any wait time for a connection and not to have to pool the connections myself - nor have to deal with any extraneous connections issues, ie. isStale ;-) What would be a good proxy for maxConnections? If it's anything less than the number of threads, wouldn't some time be spent waiting for the next available connection? Todd - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Todd Wolff wrote: Mike, that certainly makes sense. My intent was to eliminate any wait time for a connection and not to have to pool the connections myself - nor have to deal with any extraneous connections issues, ie. isStale ;-) I hear you. It is certainly okay to do it this way. I just wanted to make sure this was intentional. Also, if you were hitting more than one host there would be some added benefit to the MTCM. What would be a good proxy for maxConnections? If it's anything less than the number of threads, wouldn't some time be spent waiting for the next available connection? I really depends on a lot of factors. I would suggest limiting it at some point (I usually set it to 4). Though some methods have to wait if maxConnectionsPerHost numThreads, it also means that the executing methods will probably finish faster. The thing to keep in mind is that the sever has to handle all of the requests. At some point the overhead of the connections will outweight the benefit of concurrency. Doing a few real-world tests with your client/server configuration would probably be good. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: performance issue
Well ... there must be something different about my environment. The results with JBoss3.2.1-tomcat4.1.24 were much worse! (I decreased threads to 5 and messages per thread to 10 since tomcat defaults max threads per servlet somewhere and I didn't want to dig for the setting.) alpha3 .034 minutes beta2 .167 minutes Could it be the jdk? I'm running 1.4.0 on XP Pro service pack 1. I could try downloading and installing the most recent version of the jdk, running the test on a different machine ... hm. Also, I'm using the binaries off of the site rather than compiling from source against the jdk on my machine. Unfortunately, I won't have time to work on this any further until later this evening :-( Todd mail2web - Check your email from the web at http://mail2web.com/ . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: performance issue
It does appear to JDK 1.4.0. I just installed 1.4.0_04 on Windows 2000 and beta2 is significantly slower (almost by a factor of 10). Removing the isStale() check fixes the problem in this case. It seems 1.4.0 does not do well with SO_TIMEOUT. This does not seem to be a problem with Sun 1.3.1 and 1.4.1. or IBM 1.3.0. Mike [EMAIL PROTECTED] wrote: Well ... there must be something different about my environment. The results with JBoss3.2.1-tomcat4.1.24 were much worse! (I decreased threads to 5 and messages per thread to 10 since tomcat defaults max threads per servlet somewhere and I didn't want to dig for the setting.) alpha3 .034 minutes beta2 .167 minutes Could it be the jdk? I'm running 1.4.0 on XP Pro service pack 1. I could try downloading and installing the most recent version of the jdk, running the test on a different machine ... hm. Also, I'm using the binaries off of the site rather than compiling from source against the jdk on my machine. Unfortunately, I won't have time to work on this any further until later this evening :-( Todd mail2web - Check your email from the web at http://mail2web.com/ . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: performance issue
Take a look at Sun BUG ID 4512028. This bug was introduced in 1.4.0 and was fixed in 1.4.1. It appears to effect only Windows. Mike Michael Becke wrote: It does appear to JDK 1.4.0. I just installed 1.4.0_04 on Windows 2000 and beta2 is significantly slower (almost by a factor of 10). Removing the isStale() check fixes the problem in this case. It seems 1.4.0 does not do well with SO_TIMEOUT. This does not seem to be a problem with Sun 1.3.1 and 1.4.1. or IBM 1.3.0. Mike [EMAIL PROTECTED] wrote: Well ... there must be something different about my environment. The results with JBoss3.2.1-tomcat4.1.24 were much worse! (I decreased threads to 5 and messages per thread to 10 since tomcat defaults max threads per servlet somewhere and I didn't want to dig for the setting.) alpha3 .034 minutes beta2 .167 minutes Could it be the jdk? I'm running 1.4.0 on XP Pro service pack 1. I could try downloading and installing the most recent version of the jdk, running the test on a different machine ... hm. Also, I'm using the binaries off of the site rather than compiling from source against the jdk on my machine. Unfortunately, I won't have time to work on this any further until later this evening :-( Todd mail2web - Check your email from the web at http://mail2web.com/ . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: performance issue
Gosh, i'm sorry you guys wasted your time on what ended up being a platform specific issue. But, I guess this could be good information for someone, somewhere, including me ;-) Todd - Original Message - From: Michael Becke [EMAIL PROTECTED] To: Commons HttpClient Project [EMAIL PROTECTED] Sent: Tuesday, July 29, 2003 8:37 AM Subject: Re: performance issue It does appear to JDK 1.4.0. I just installed 1.4.0_04 on Windows 2000 and beta2 is significantly slower (almost by a factor of 10). Removing the isStale() check fixes the problem in this case. It seems 1.4.0 does not do well with SO_TIMEOUT. This does not seem to be a problem with Sun 1.3.1 and 1.4.1. or IBM 1.3.0. Mike [EMAIL PROTECTED] wrote: Well ... there must be something different about my environment. The results with JBoss3.2.1-tomcat4.1.24 were much worse! (I decreased threads to 5 and messages per thread to 10 since tomcat defaults max threads per servlet somewhere and I didn't want to dig for the setting.) alpha3 .034 minutes beta2 .167 minutes Could it be the jdk? I'm running 1.4.0 on XP Pro service pack 1. I could try downloading and installing the most recent version of the jdk, running the test on a different machine ... hm. Also, I'm using the binaries off of the site rather than compiling from source against the jdk on my machine. Unfortunately, I won't have time to work on this any further until later this evening :-( Todd mail2web - Check your email from the web at http://mail2web.com/ . - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
On Monday, July 28, 2003, at 06:10 AM, Kalnichevski, Oleg wrote: I would just make 'stale' connection check optional and off per default. Making the check pluggable would require too much of a change for what is essentially a release candidate. That should be better done as a part of 3.0 redesign IMO. I agree that removing isStale() by default certainly fixes the performance problem. However, it makes MTCM somewhat unstable. The case where stale connections do not cause errors until read from will throw unrecoverable exceptions. Is this an acceptable alternative? We definitely should. I consider myself fairly knowledgeable about HttpClient internals, but I have to say this massage has caught me by surprise. I would also consider you quite knowledgeable about HttpClient internals:) I will look to see what we can do here. I would also really like to see a few more DEBUG logs to be added to the MultiThreadedHttpConnectionManager#doGetConnection method Okay. Would you like to take care of this or should I? Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Performance Issue
I agree that removing isStale() by default certainly fixes the performance problem. However, it makes MTCM somewhat unstable. The case where stale connections do not cause errors until read from will throw unrecoverable exceptions. Is this an acceptable alternative? What if we keep 'IsStale' check on per default for multi-threaded connection manager and off for the single-threaded one? Okay. Would you like to take care of this or should I? I think as you authored the MTCM in the first place you can do it better. Would that be ok with you? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Kalnichevski, Oleg wrote: I would just make 'stale' connection check optional and off per default. Making the check pluggable would require too much of a change for what is essentially a release candidate. That should be better done as a part of 3.0 redesign IMO. As I recall, the isStale function solved a problem arose when the server closed its write channel, but not its read channel. HttpClient would then send a request, and only when it went to read the response would it fail. Some possible alternatives: * Only do the isStale check if the connection has been sitting in the pool for a configurable amount of time. I'm guessing we could choose a value here between 5 and 30 seconds without any significant change in behavior, that is to say, connections won't go stale in less than 20-30s. * Perhaps the isStale check is unnecessary for methods that can simply repeat themselves, for example, GET, HEAD, OPTION, TRACE. For those methods, we could allow a retry to fix the problem. For methods such as POST and PUT, however, the isStale is probably an essential check. * Is this a confirmed problem across all VMs and all OSes? Is this a confirmed problem if not invoking localhost? If it affects one platform, could we punt on the issue? Which is the specific line in isStale() that causes the performance degradation? Is there anyway to speed that one line? -Eric. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Eric Johnson wrote: Some possible alternatives: * Only do the isStale check if the connection has been sitting in the pool for a configurable amount of time. I'm guessing we could choose a value here between 5 and 30 seconds without any significant change in behavior, that is to say, connections won't go stale in less than 20-30s. Not agreed here. A connection can become stale at virtually any time due to the underlying IP stack implementation, or misbehaving servers. As seen in the wild. * Perhaps the isStale check is unnecessary for methods that can simply repeat themselves, for example, GET, HEAD, OPTION, TRACE. For those methods, we could allow a retry to fix the problem. For methods such as POST and PUT, however, the isStale is probably an essential check. Sounds reasonable to me. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Todd, I believe we have identified the reason for performance degradation which is a stale connection check introduced in beta-1, not the multi-threaded connection manager itself. The bogus log entries about connection being created are caused by the connection wrapper class, which is indeed created every time a connection is obtained from the connection pool. This said, you may still send your test application but I believe we can trust Mike on his word that multi-threaded connection manager is not the problem Cheers Oleg On Mon, 2003-07-28 at 17:45, Todd Wolff wrote: Glad I could contribute. I've put together a test harness that reproduces the connection issue. Unless I'm missing something, I'm pretty sure only a single instance of MTCM is being accessed by the sending threads. Should I send it to you and Mike directly rather than taking up bandwidth on the list (it's 250k.) Todd - Original Message - From: Oleg Kalnichevski [EMAIL PROTECTED] To: Commons HttpClient Project [EMAIL PROTECTED] Sent: Sunday, July 27, 2003 7:10 AM Subject: Re: Performance Issue Todd, I think I have identified the problem or one of the problem that contributed to the noticeable performance degradation for short HTTP messages in post 2.0a3 releases. So, many thanks for bringing up this issue Further, see my comments below On Sun, 2003-07-27 at 03:36, Todd Wolff wrote Also, although unrelated to the relative decrease in performance, I did notice that in both tests a new connection is created per request. Usually HttpClient does a pretty good job reusing connections. You can see the evidence of that from the log that you have produced: ... [DEBUG] HttpMethodBase - -Resorting to protocol version default close connection policy [DEBUG] HttpMethodBase - -Should NOT close connection, using HTTP/1.1 [DEBUG] MultiThreadedHttpConnectionManager - -Freeing connection: ... What is fishy here is why the connection manager keeps on creating new connections instead of reusing those active ones that have been just returned to the pool. I can't tell. My guess is that you end up creating an instance of MultiThreadedHttpConnectionManager per thread in your application that completely defeats its purpose. You should be sharing the same instance of mtcm between all your worker threads in order to make connection pooling effective. This is just a guess, though. You'll have to wait until Michael Becke gets on-line. He's the one who wrote MultiThreadedHttpConnectionManager in the first place, so he will be of MUCH more help here. Mike, we clearly need more DEBUG logs in the MultiThreadedHttpConnectionManager. At the moment it is difficult to tell what is going on there. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
* Perhaps the isStale check is unnecessary for methods that can simply repeat themselves, for example, GET, HEAD, OPTION, TRACE. For those methods, we could allow a retry to fix the problem. For methods such as POST and PUT, however, the isStale is probably an essential check. Sounds reasonable to me. A clear drawback to this approach is that the connection would need to 'know' what kind of request is being executed. Of course, this is just design purism. Feel free to ignore me. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Just sending a quick note before I go to bed. I did some testing with the example code and am now quite confident that HTCM is doing what it should. I have not had time to add any DEBUG code but hopefully I will get to it tomorrow. Todd, one thing I noticed is that you are setting maxConnectionsPerHost to the number of threads you are creating. This pretty much defeats the purpose of using the MTCM and there is no need to share connections. There are enough connections for each thread to just own one. This is not really related to the problem with isStale() but I thought I would mention it. Oleg, have you done any timings with Todd's example code? I have found that disabling isStale() is this case seems to have little effect on execution time. I was not able to test against 2.0 alpha3 and am wondering if it would make any difference. Mike On Monday, July 28, 2003, at 11:45 AM, Todd Wolff wrote: Glad I could contribute. I've put together a test harness that reproduces the connection issue. Unless I'm missing something, I'm pretty sure only a single instance of MTCM is being accessed by the sending threads. Should I send it to you and Mike directly rather than taking up bandwidth on the list (it's 250k.) Todd - Original Message - From: Oleg Kalnichevski [EMAIL PROTECTED] To: Commons HttpClient Project [EMAIL PROTECTED] Sent: Sunday, July 27, 2003 7:10 AM Subject: Re: Performance Issue Todd, I think I have identified the problem or one of the problem that contributed to the noticeable performance degradation for short HTTP messages in post 2.0a3 releases. So, many thanks for bringing up this issue Further, see my comments below On Sun, 2003-07-27 at 03:36, Todd Wolff wrote Also, although unrelated to the relative decrease in performance, I did notice that in both tests a new connection is created per request. Usually HttpClient does a pretty good job reusing connections. You can see the evidence of that from the log that you have produced: ... [DEBUG] HttpMethodBase - -Resorting to protocol version default close connection policy [DEBUG] HttpMethodBase - -Should NOT close connection, using HTTP/1.1 [DEBUG] MultiThreadedHttpConnectionManager - -Freeing connection: ... What is fishy here is why the connection manager keeps on creating new connections instead of reusing those active ones that have been just returned to the pool. I can't tell. My guess is that you end up creating an instance of MultiThreadedHttpConnectionManager per thread in your application that completely defeats its purpose. You should be sharing the same instance of mtcm between all your worker threads in order to make connection pooling effective. This is just a guess, though. You'll have to wait until Michael Becke gets on-line. He's the one who wrote MultiThreadedHttpConnectionManager in the first place, so he will be of MUCH more help here. Mike, we clearly need more DEBUG logs in the MultiThreadedHttpConnectionManager. At the moment it is difficult to tell what is going on there. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
On Sunday, July 27, 2003, at 12:39 PM, Oleg Kalnichevski wrote: I suppose this particular problem is not related to isStale(), as I do not see in the logs any connections dropped because they appear to be 'stale'. I still lean toward assuming this is an issue with Todd' code, rather HttpClient itself. I think the only problem is with HttpConnection.isStale(). I have done some timings and can confirm that isSale() significantly increases method execution time. In fact, when connecting to localhost, it is actually faster to create a new connection for each method execution than to reuse connections. This is obviously a bad thing. I believe our only real option is to make isStale() optional and/or pluggable. Effective alternatives to isStale() have been thoroughly investigated and I think we can say that they do not exist, at least in a pre 1.4 environment. Does anyone have some new insight into this problem? I would like to suggest that the problems are dealt with sequentially: - fix isStale performance degradation problem. In the worse case, we can simply make stale connection check optional, if we do not come up with something better - Retag HTTPCLIENT_2_0_RC1 - Get Todd upgrade to HTTPCLIENT_2_0_RC1 - See if we get more information about what it going on in the MTCM As stated previously I do not think there is anything wrong with the MTCM. My guess is that the misunderstanding is being caused by the following DEBUG messages: [java] [DEBUG] HttpConnection - -Creating connection for localhost using protocol http:80 These messages appear every time a connection is retrieved from the MTCM. This is because of the following line in MTCM.getConnection(HostConfiguration, long): // wrap the connection in an adapter so we can ensure it is used // only once return new HttpConnectionAdapter(conn); This connection adapter is created each time and is causing the DEBUG messages ( HttpConnection(String, int, String, String, int, Protocol) is the source). Perhaps we should find some way to remove these spurious messages. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
I think I tell the reason even without having seen the code. Since beta-1 'expect: 100-continue' handshake is off per default, that can make a huge difference in terms of performance with POST requests that require authentication. Just do the following httppost.setUseExpectHeader(true); http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/methods/ExpectContinueMethod.html Alternatively you could use preemptive authentication. I hope this helps Cheers Oleg On Sat, 2003-07-26 at 05:35, Todd Wolff wrote: Hi, After upgrading from 2.0-alpha3 to 2.0-beta2, instead of roughly 10 requests per second, I am averaging only 3 requests per second. I was hoping someone could take a look at the attached code and show me the 'error of my ways.' My test is multithreaded, and all requests are sent to the same host. I am setting MaxConnectionsPerHost equivalent to the number of sending threads. The auth-method required by the server is BASIC. Thanks for your help. Todd __ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance Issue
Thanks for the reply Oleg. I tried setUseExpectHeader(true) as suggested,
but no improvement. I ran the exact same test with all else held equal
except the httpclient libs used (alpha3 vs. beta2) and I've attached the
debug logs. I've also attached the code. (No security constraint was used
this time, so authorization shouldn't have anything to do with it.)
I've been looking at this all day and can't see the reason for the
difference. Also, although unrelated to the relative decrease in
performance, I did notice that in both tests a new connection is created per
request. Is there anything I can do to encourage re-use?
Gracias,
Todd
- Original Message -
From: Oleg Kalnichevski [EMAIL PROTECTED]
To: Commons HttpClient Project [EMAIL PROTECTED]
Sent: Saturday, July 26, 2003 1:48 AM
Subject: Re: Performance Issue
I think I tell the reason even without having seen the code. Since
beta-1 'expect: 100-continue' handshake is off per default, that can
make a huge difference in terms of performance with POST requests that
require authentication.
Just do the following
httppost.setUseExpectHeader(true);
http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/http
client/methods/ExpectContinueMethod.html
Alternatively you could use preemptive authentication.
I hope this helps
Cheers
Oleg
On Sat, 2003-07-26 at 05:35, Todd Wolff wrote:
Hi,
After upgrading from 2.0-alpha3 to 2.0-beta2, instead of roughly 10
requests per second, I am averaging only 3 requests per second. I was
hoping someone could take a look at the attached code and show me the
'error of my ways.' My test is multithreaded, and all requests are
sent to the same host. I am setting MaxConnectionsPerHost equivalent
to the number of sending threads. The auth-method required by the
server is BASIC.
Thanks for your help.
Todd
__
-
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
-
To unsubscribe, e-mail:
[EMAIL PROTECTED]
For additional commands, e-mail:
[EMAIL PROTECTED]
import java.util.Properties;
import javax.servlet.http.*;
import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.methods.*;
import org.apache.commons.logging.Log;
import com.bluestem.alakai.common.exception.AIException;
import com.bluestem.alakai.common.util.Logger;
public class XmlHTTPClient {
private static Log log = Logger.getLog(XmlHTTPClient.class);
private Properties props = null;
private String strURL = null;
private int timeout = 30; //five minute default
private String userName = null;
private String password = null;
private boolean authenticationRequired;
private boolean handleAuthentication;
private int connections = 5; // default to 5
private HttpClient httpClient = null;
public XmlHTTPClient(Properties props) {
this.props = props;
}
public synchronized void start() throws AIException {
Logger.debug(log, start begin);
parseProps();
MultiThreadedHttpConnectionManager mthcm = new
MultiThreadedHttpConnectionManager();
mthcm.setMaxConnectionsPerHost(connections);
httpClient = new HttpClient(mthcm);
httpClient.setTimeout(timeout);
if (authenticationRequired) {
httpClient.getState().setCredentials(
null,
new UsernamePasswordCredentials(userName, password)
);
// send the authentication before the challenge is received
httpClient.getState().setAuthenticationPreemptive(true);
}
Logger.debug(log, started with the following parameters: + \n +
strURL + strURL + \n +
timeout + timeout + \n +
userName + userName + \n +
password + password + \n +
handleAuthentication + handleAuthentication + \n +
pooledConnections + connections
);
}
public String sendMessage(String requestBody) throws AIException {
Logger.debug(log, sending request);
String xmlResponse = null;
PostMethod httppost = null;
try {
// create the postmethod using the url passed into constructor
httppost = new PostMethod(strURL);
// Tell the post method to automatically handle authentication. The method
will use
// any appropriate credentials to handle basic authentication requests.
Setting this
// value to false will cause any request for authentication to return with
a status
// of 401. It will then be up
