Re: article draft - Summary of benchmark

[Remy Maucherat]

On Tue, 1 Feb 2005 15:18:59 -0500, Peter Lin <[EMAIL PROTECTED]> wrote:
> I've updated the document with more charts from the excel spreadsheet
> and tried to make the explanations more clear.

I'm really interested in the part of your tests which show certain new
CPU architectures showing a big improvement in Java. I suppose it
benefits a lot from large caches (Pentium M, Opteron), and I suppose
more regirsters won't hurt either (x86-64).

-- 
x
Rémy Maucherat
Developer & Consultant
JBoss Group (Europe) SàRL
x

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: article draft - Summary of benchmark

[Peter Lin]

I've updated the document with more charts from the excel spreadsheet
and tried to make the explanations more clear.

peter


On Mon, 31 Jan 2005 14:20:11 -0500, Peter Lin <[EMAIL PROTECTED]> wrote:
> http://cvs.apache.org/~woolfel/benchmark_summary.doc
> http://cvs.apache.org/~woolfel/benchmark_summary.sxw
> http://cvs.apache.org/~woolfel/tc_results.html
> 
> I've finished a complete draft of the benchmark results. If you have
> any feedback, please email me directly. thanks in advance.
> 
> 
> peter lin
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

article draft - Summary of benchmark

[Peter Lin]

http://cvs.apache.org/~woolfel/benchmark_summary.doc
http://cvs.apache.org/~woolfel/benchmark_summary.sxw
http://cvs.apache.org/~woolfel/tc_results.html

I've finished a complete draft of the benchmark results. If you have
any feedback, please email me directly. thanks in advance.


peter lin

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Some charts of benchmark results

[Peter Lin]

Here are some charts for the benchmark results I posted a few days
back. It's taking longer than I expected to create the charts.

http://cvs.apache.org/~woolfel/tc_results.html

enjoy.

peter

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: latest set of Benchmark results

[Peter Lin]

quick explanation of errors for 40K png test. The errors were the
result of OS and not Tomcat. For some odd reason, after a couple of
hours of benchmarks, the OS stopped accepting connections. I will
re-run that one test later for my final write up.

I've posted updated testplans including 4 with gzip header.

http://cvs.apache.org/~woolfel/testplans.zip

peter lin

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: latest set of Benchmark results

[Remy Maucherat]

On Sun, 16 Jan 2005 13:15:01 -0500, Peter Lin <[EMAIL PROTECTED]> wrote:
> Here is the latest set of benchmark results. I discovered an error in
> my test plan for 40K PNG, so the results for that one was off. All of
> the other results should be accurate. I re-ran the tests.
> 
> Server:
> AMD 2ghz
> 1Gb RAM
> Redhat FedoraCore1
> 
> Client:
> Gateway laptop 450
> 1.4ghz centrino
> Windows XP pro
> jdk1.4.2
> jmeter nightly build
> 
> 16 port Switch Linksys 10/100
> CAT5 cables
> 
> All threads ran for 1000 iterations.
> thread scenario: 5, 10, 15, 20, 25, 30
> image/html size: 1, 10, 20, 40, 80, 160
> 
> The test includes 5.0.19 (aka 5.0), 5.5.4, apache 2.0.50, apache
> 1.3.3.  I hope people find it interesting and useful. It will take me
> a week or two to write up the results and generate the graphs.

Does the "-server" option do anything other than use more memory ? ;)

-- 
x
Rémy Maucherat
Developer & Consultant
JBoss Group (Europe) SàRL
x

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

latest set of Benchmark results

[Peter Lin]

Here is the latest set of benchmark results. I discovered an error in
my test plan for 40K PNG, so the results for that one was off. All of
the other results should be accurate. I re-ran the tests.


Server:
AMD 2ghz
1Gb RAM
Redhat FedoraCore1

Client:
Gateway laptop 450
1.4ghz centrino
Windows XP pro
jdk1.4.2
jmeter nightly build

16 port Switch Linksys 10/100
CAT5 cables

All threads ran for 1000 iterations.
thread scenario: 5, 10, 15, 20, 25, 30
image/html size: 1, 10, 20, 40, 80, 160

The test includes 5.0.19 (aka 5.0), 5.5.4, apache 2.0.50, apache
1.3.3.  I hope people find it interesting and useful. It will take me
a week or two to write up the results and generate the graphs.


peter lin




Tomcat 5.0.19 - jdk1.4.2
PNG
1K
---
protocol | samples | average | median | 90% line | min | max | error%
| throughput | Kb/sec
---
HTTP Request50005.1556  0   10  0   681 0.00%   
433.0/sec   446.54
HTTP Request1   6.5027  0   10  0   17020.00%   
813.1/sec   838.55
HTTP Request15000   4.83573 0   10  0   150 0.00%   
1333.8/sec  1375.49
HTTP Request2   5.0478  0   10  0   20030.00%   
1236.6/sec  1275.19
HTTP Request25000   11.1284 10  30  0   260 0.00%   
1258.9/sec  1298.28
HTTP Request3   21.2057 20  30  0   100 0.00%   
1221.2/sec  1259.41

5k

HTTP Request50003.3992  0   10  0   270 0.00%   
894.8/sec   4552.51
HTTP Request1   5.5172  0   10  0   421 0.00%   
973.2/sec   4951.72
HTTP Request15000   5.47953 0   20  0   201 0.00%   
891.0/sec   4533.32
HTTP Request2   8.15755 0   20  0   291 0.00%   
929.8/sec   4730.50
HTTP Request25000   12.5837 10  30  0   190 0.00%   
921.9/sec   4690.34
HTTP Request3   11.5905 0   40  0   15220.00%   
822.5/sec   4184.92

10k

HTTP Request50004.4988  0   10  0   15730.00%   
749.6/sec   7538.71
HTTP Request1   8.2505  10  20  0   230 0.00%   
766.3/sec   7706.83
HTTP Request15000   6.98413 0   20  0   862 0.00%   
727.4/sec   7315.69
HTTP Request2   12.345  10  30  0   300 0.00%   
710.7/sec   7147.32
HTTP Request25000   18.6458 11  40  0   430 0.00%   
696.7/sec   7006.74
HTTP Request3   33.4359 40  60  0   300 0.00%   
680.7/sec   6845.44

20k

HTTP Request500010.8294 0   10  0   30840.00%   
259.0/sec   5231.30
HTTP Request1   14.9086 10  30  0   13720.00%   
418.1/sec   8447.07
HTTP Request15000   22.2814 20  40  0   471 0.00%   
490.5/sec   9907.72
HTTP Request2   37.3174 40  50  0   440 0.00%   
491.2/sec   9922.48
HTTP Request25000   37.4917 40  70  0   15520.00%   
448.0/sec   9050.54
HTTP Request3   43.7394 50  80  0   23930.00%   
452.8/sec   9146.87

40k

HTTP Request500016.1594 11  20  0   270 0.00%   
270.2/sec   10825.45
HTTP Request1   32.0691 30  40  0   281 0.00%   
283.8/sec   11370.09
HTTP Request15000   49.0089 50  60  0   231 0.00%   
285.1/sec   11425.53
HTTP Request2   62.9201 60  80  0   461 0.00%   
284.7/sec   11405.85
HTTP Request25000   76.398  80  100 0   721 0.00%   
279.6/sec   11204.07
HTTP Request3   100.121 100 140 0   831 0.00%   
272.7/sec   10927.21

80k

HTTP Request500033.5142 30  40  0   310 0.00%   
141.2/sec   11349.39
HTTP Request1   67.5162 70  80  0   280 0.00%   
143.0/sec   11495.76
HTTP Request15000   103.413 100 111 0   811 0.00%   
140.1/sec   11261.33
HTTP Request2   141.583 130 150 0   942 0.00%   
137.9/sec   11090.00
HTTP Request25000   178.314 170 191 0   11820.00%   
136.2/sec   10951.84
HTTP Request3   215.295 191 270 0   13120.00%   
135.5/sec   10897.99

160k

HTTP Request500067.4416 70  71  10  200 0.00%   
71.9/sec11509.28
HTTP Request1   137.940 131 141 10  901 0.00%   
71.0/sec11367.65
HTTP Request15000   211.484 200 221 10  10820.00%   
69.6/sec11131.86
HTTP Request2   287.701 261 330

Re: more benchmark results

[Didier McGillis]

I run FC3, FC2, RH9 and soon will have Debain install going, I would be 
happy to help or whatever.

From: "Sessoms, Mack" <[EMAIL PROTECTED]>
Reply-To: "Tomcat Users List" 
To: Tomcat Users List 
Subject: Re: more benchmark results
Date: Fri, 14 Jan 2005 09:28:52 -0500
i've got 5.5.4 w/jvm 1.5.0-b64 on fc3 (2.6.9-1.667,i386).  unfortunately, 
the hardware is a desktop unit 760 MB ram, ide drive, 2.6 P4 (512 cache).  
let me know if i should give it a try.

Peter Lin wrote:
any tomcat user out there have Redhat FC3 installed and want to help
run some tests?  I will post the jmeter test plans this weekend.
it would make remy really happy :)
peter


And no FC 3 ? ;) I think it would run fine on your computer, and it's
a higher quality distribution overall (it doesn't have the stupid NPTL
backport that FC 1 has). Or you could try Ubuntu (I plan to switch to
that distro when they release hoary).
Anyway, I'd be interested if you tried stressing a little the thread
pool I added in 5.5 (strategy="ms" threadPriority="7" on the Connector
element) to see if it gives a difference in the error rate (and also
if it's not completely broken). You may want to increase maxThreads as
well for your tests (it's 150 by default, which is dangerously close
of the concurrency used by your client)
--
x
Rémy Maucherat
Developer & Consultant
JBoss Group (Europe) SàRL
x

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Peter Lin]

for those who want to look at the test plans or assist, I've posted
the jmeter test plans

http://cvs.apache.org/~woolfel/testplans.zip

peter


On Fri, 14 Jan 2005 09:28:52 -0500, Sessoms, Mack <[EMAIL PROTECTED]> wrote:
> i've got 5.5.4 w/jvm 1.5.0-b64 on fc3 (2.6.9-1.667,i386).
> unfortunately, the hardware is a desktop unit 760 MB ram, ide drive, 2.6
> P4 (512 cache).  let me know if i should give it a try.
> 
> Peter Lin wrote:
> 
> >any tomcat user out there have Redhat FC3 installed and want to help
> >run some tests?  I will post the jmeter test plans this weekend.
> >
> >it would make remy really happy :)
> >
> >peter
> >
> >
> >
> >
> >>And no FC 3 ? ;) I think it would run fine on your computer, and it's
> >>a higher quality distribution overall (it doesn't have the stupid NPTL
> >>backport that FC 1 has). Or you could try Ubuntu (I plan to switch to
> >>that distro when they release hoary).
> >>
> >>Anyway, I'd be interested if you tried stressing a little the thread
> >>pool I added in 5.5 (strategy="ms" threadPriority="7" on the Connector
> >>element) to see if it gives a difference in the error rate (and also
> >>if it's not completely broken). You may want to increase maxThreads as
> >>well for your tests (it's 150 by default, which is dangerously close
> >>of the concurrency used by your client)
> >>
> >>--
> >>x
> >>Rémy Maucherat
> >>Developer & Consultant
> >>JBoss Group (Europe) SàRL
> >>x
> >>
> >>
> >>
> >
> >-
> >To unsubscribe, e-mail: [EMAIL PROTECTED]
> >For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
> >
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Peter Lin]

Sure, it can't hurt to try it out. Even if the specs are different,
the data point is a good baseline comparison. I'll post the test plans
shortly to my apache directory.

peter



On Fri, 14 Jan 2005 09:28:52 -0500, Sessoms, Mack <[EMAIL PROTECTED]> wrote:
> i've got 5.5.4 w/jvm 1.5.0-b64 on fc3 (2.6.9-1.667,i386).
> unfortunately, the hardware is a desktop unit 760 MB ram, ide drive, 2.6
> P4 (512 cache).  let me know if i should give it a try.
> 
> Peter Lin wrote:
> 
> >any tomcat user out there have Redhat FC3 installed and want to help
> >run some tests?  I will post the jmeter test plans this weekend.
> >
> >it would make remy really happy :)
> >
> >peter
> >
> >
> >
> >
> >>And no FC 3 ? ;) I think it would run fine on your computer, and it's
> >>a higher quality distribution overall (it doesn't have the stupid NPTL
> >>backport that FC 1 has). Or you could try Ubuntu (I plan to switch to
> >>that distro when they release hoary).
> >>
> >>Anyway, I'd be interested if you tried stressing a little the thread
> >>pool I added in 5.5 (strategy="ms" threadPriority="7" on the Connector
> >>element) to see if it gives a difference in the error rate (and also
> >>if it's not completely broken). You may want to increase maxThreads as
> >>well for your tests (it's 150 by default, which is dangerously close
> >>of the concurrency used by your client)
> >>
> >>--
> >>x
> >>Rémy Maucherat
> >>Developer & Consultant
> >>JBoss Group (Europe) SàRL
> >>x
> >>
> >>
> >>
> >
> >-
> >To unsubscribe, e-mail: [EMAIL PROTECTED]
> >For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
> >
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Sessoms, Mack]

i've got 5.5.4 w/jvm 1.5.0-b64 on fc3 (2.6.9-1.667,i386).  
unfortunately, the hardware is a desktop unit 760 MB ram, ide drive, 2.6 
P4 (512 cache).  let me know if i should give it a try.

Peter Lin wrote:
any tomcat user out there have Redhat FC3 installed and want to help
run some tests?  I will post the jmeter test plans this weekend.
it would make remy really happy :)
peter
 

And no FC 3 ? ;) I think it would run fine on your computer, and it's
a higher quality distribution overall (it doesn't have the stupid NPTL
backport that FC 1 has). Or you could try Ubuntu (I plan to switch to
that distro when they release hoary).
Anyway, I'd be interested if you tried stressing a little the thread
pool I added in 5.5 (strategy="ms" threadPriority="7" on the Connector
element) to see if it gives a difference in the error rate (and also
if it's not completely broken). You may want to increase maxThreads as
well for your tests (it's 150 by default, which is dangerously close
of the concurrency used by your client)
--
x
Rémy Maucherat
Developer & Consultant
JBoss Group (Europe) SàRL
x
   

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
 

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Larry Meadors]

On Fri, 14 Jan 2005 09:07:37 -0500, Peter Lin <[EMAIL PROTECTED]> wrote:
> what a concept Schnapps over speed. Wonder what would happen if
> someone made a redbull + Schnapps + speed cocktail 8-)

Use it to chase the diet pills and nodoz, and you may never sleep again. :-)

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Peter Lin]

what a concept Schnapps over speed. Wonder what would happen if
someone made a redbull + Schnapps + speed cocktail 8-)

that wouldn't kill you, really.

I'll stop the jokes there.

peter

On Fri, 14 Jan 2005 14:59:57 +0100, Mladen Turk <[EMAIL PROTECTED]> wrote:
> Peter Lin wrote:
> >
> > If I have time, I also plan to compare httpd 1.3.  All I need now is
> > some speed so I can go without sleep :)
> >
> 
> In my country we are using the thing called 'Rakija' for that.
> It has 45% of alcohol, but has some strange side-effects.
> Someone are even stating that it can make you drunk !?
> 
> So, I suggest Rakija
> (or as my German friends are wrongly calling it Schnapps)
> over speed :).
> 
> Mladen.
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Mladen Turk]

Peter Lin wrote:
If I have time, I also plan to compare httpd 1.3.  All I need now is
some speed so I can go without sleep :)
In my country we are using the thing called 'Rakija' for that.
It has 45% of alcohol, but has some strange side-effects.
Someone are even stating that it can make you drunk !?
So, I suggest Rakija
(or as my German friends are wrongly calling it Schnapps)
over speed :).
Mladen.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Peter Lin]

yeah, I've d/l 2.0.52 and plan on running a test. Graham O'Regan is
also going to run the tests again Squid for comparison too.

If I have time, I also plan to compare httpd 1.3.  All I need now is
some speed so I can go without sleep :)

peter


On Fri, 14 Jan 2005 14:40:16 +0100, Mladen Turk <[EMAIL PROTECTED]> wrote:
> Peter Lin wrote:
> > Yup, I plan to try jdk5 with TC5.5.4 per Remy's request.
> >
> 
> Cool, but can you compare the results with Apache2.0.52
> when serving the same static content files on the same
> hardware?
> 
> That would be very interesting thought. Even more then
> bare statistical data you've presented.
> 
> Regards,
> Mladen
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Peter Lin]

any tomcat user out there have Redhat FC3 installed and want to help
run some tests?  I will post the jmeter test plans this weekend.

it would make remy really happy :)

peter


> And no FC 3 ? ;) I think it would run fine on your computer, and it's
> a higher quality distribution overall (it doesn't have the stupid NPTL
> backport that FC 1 has). Or you could try Ubuntu (I plan to switch to
> that distro when they release hoary).
> 
> Anyway, I'd be interested if you tried stressing a little the thread
> pool I added in 5.5 (strategy="ms" threadPriority="7" on the Connector
> element) to see if it gives a difference in the error rate (and also
> if it's not completely broken). You may want to increase maxThreads as
> well for your tests (it's 150 by default, which is dangerously close
> of the concurrency used by your client)
> 
> --
> x
> Rémy Maucherat
> Developer & Consultant
> JBoss Group (Europe) SàRL
> x
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Mladen Turk]

Peter Lin wrote:
Yup, I plan to try jdk5 with TC5.5.4 per Remy's request.
Cool, but can you compare the results with Apache2.0.52
when serving the same static content files on the same
hardware?
That would be very interesting thought. Even more then
bare statistical data you've presented.
Regards,
Mladen

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Remy Maucherat]

On Fri, 14 Jan 2005 08:11:11 -0500, Peter Lin <[EMAIL PROTECTED]> wrote:
> Per Remy's request, I ran some more tests last night with larger
> number of threads. the configuration of the test plan is as follows
> 
> 1K png: 10, 50, 100, 150 threads
> 10K png: 10, 50, 100, 150 threads
> 
> each thread as was to 1000 iterations.
> ramp up times: 1, 5, 10, 20 seconds
> 
> Server:
> Redhat fedora Core1
> AMD 2ghz
> 1Gb RAM
> tomcat 5.0.19
> Sun jdk1.4.2_03
> 
> Client:
> gateway 450 laptop
> 1.4ghz centrino
> 1Gb RAM
> jmeter nightly build
> sun jdk1.4.2
> 
> I plan to re-run these tests with the latest 5.0.x release this
> weekend and per Remy's request I will also test 5.5.4 with jdk5 for
> comparison.

And no FC 3 ? ;) I think it would run fine on your computer, and it's
a higher quality distribution overall (it doesn't have the stupid NPTL
backport that FC 1 has). Or you could try Ubuntu (I plan to switch to
that distro when they release hoary).

Anyway, I'd be interested if you tried stressing a little the thread
pool I added in 5.5 (strategy="ms" threadPriority="7" on the Connector
element) to see if it gives a difference in the error rate (and also
if it's not completely broken). You may want to increase maxThreads as
well for your tests (it's 150 by default, which is dangerously close
of the concurrency used by your client)

-- 
x
Rémy Maucherat
Developer & Consultant
JBoss Group (Europe) SàRL
x

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Peter Lin]

Yup, I plan to try jdk5 with TC5.5.4 per Remy's request.

once I get that out of the way, I plan to work on the enhancements to
the status servlet. I took a quick look at apr-java, looks cool. No
comments yet, but I will post them to tomcat-dev when I do :)

peter



On Fri, 14 Jan 2005 14:30:38 +0100, Mladen Turk <[EMAIL PROTECTED]> wrote:
> Peter Lin wrote:
> > Per Remy's request, I ran some more tests last night with larger
> > number of threads. the configuration of the test plan is as follows
> >
> 
> What would be nice (since you have infrastructure set up)
> is to compare the results with Apache2.
> Last time i did it there was only 10% difference when
> serving static content.
> 
> Also can you try the Java5?
> 
> Regards,
> Mladen.
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: more benchmark results

[Mladen Turk]

Peter Lin wrote:
Per Remy's request, I ran some more tests last night with larger
number of threads. the configuration of the test plan is as follows
What would be nice (since you have infrastructure set up)
is to compare the results with Apache2.
Last time i did it there was only 10% difference when
serving static content.
Also can you try the Java5?
Regards,
Mladen.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

more benchmark results

[Peter Lin]

Per Remy's request, I ran some more tests last night with larger
number of threads. the configuration of the test plan is as follows

1K png: 10, 50, 100, 150 threads
10K png: 10, 50, 100, 150 threads

each thread as was to 1000 iterations.
ramp up times: 1, 5, 10, 20 seconds

Server:
Redhat fedora Core1
AMD 2ghz
1Gb RAM
tomcat 5.0.19
Sun jdk1.4.2_03

Client:
gateway 450 laptop
1.4ghz centrino
1Gb RAM
jmeter nightly build
sun jdk1.4.2


I plan to re-run these tests with the latest 5.0.x release this
weekend and per Remy's request I will also test 5.5.4 with jdk5 for
comparison.

peter


1K
---
protocol | samples | average | median | 90% line | min | max | error%
| throughput | Kb/sec
---
HTTP Request1   2.8969  0   10  0   571 0.00%   
1678.1/sec  1730.58
HTTP Request5   6.615   0   20  0   320 0.00%   
1399.3/sec  1443.07
HTTP Request10  70.106  0   100 0   26743.98%   
829.9/sec   868.94
HTTP Request15  117.443 0   321 0   54283.88%   
704.5/sec   737.30

10k

HTTP Request1   8.2566  0   10  0   321 0.00%   
848.4/sec   8531.98
HTTP Request5   37.6230 40  81  0   28940.00%   
721.8/sec   7259.02
HTTP Request10  117.799 10  210 0   70504.12%   
524.9/sec   5092.10
HTTP Request15  297.357 300 490 0   93431.03%   
397.1/sec   3958.41

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: some TC5 benchmark results for static file

[Remy Maucherat]

On Thu, 13 Jan 2005 10:16:08 -0500, Peter Lin <[EMAIL PROTECTED]> wrote:
> I've started a series of benchmarks to measure tomcat5 performance for
> static files and compare it to apache2. Here are the results I have so
> far. I thought others might find it interesting.
> 
> Server:
> Redhat Fedora Core 1
> AMD 2hgz
> 1Gb ram
> jdk1.4.2
> TC5.0.x ( have to double check the release number)

I would like a quick comparison with the new stuff, and tests with
higher concurrency (if possible):
FC 3 + JRE 5 + Tomcat 5.5.7

Especially if you plan to write an an article, I'd be glad if you used
the best available platform :)

-- 
x
Rémy Maucherat
Developer & Consultant
JBoss Group (Europe) SàRL
x

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: some TC5 benchmark results for static file

[Peter Lin]

Good question. I haven't run the apache tests yet. I forgot that I
don't have apache2 installed on my linux box, so this weekend I plan
to install and run the tests.

once all the tests are done, I plan to write up a quick article,
generate some charts and zip up all the files, including the jmeter
log files.

peter


On Thu, 13 Jan 2005 10:45:36 -0500, Derrick Koes
<[EMAIL PROTECTED]> wrote:
> 
> So, are the results for tomcat?  Looks like you did some requests for png 
> image files and HTML.  Where are the numbers for apache so we can compare?
> 
> Thanks,
> Derrick
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: some TC5 benchmark results for static file

[Derrick Koes]

 
So, are the results for tomcat?  Looks like you did some requests for png image 
files and HTML.  Where are the numbers for apache so we can compare?

Thanks,
Derrick

-Original Message-
From: Peter Lin [mailto:[EMAIL PROTECTED] 
Sent: Thursday, January 13, 2005 10:16 AM
To: tomcat-user
Subject: some TC5 benchmark results for static file

I've started a series of benchmarks to measure tomcat5 performance for static 
files and compare it to apache2. Here are the results I have so far. I thought 
others might find it interesting.

Server:
Redhat Fedora Core 1
AMD 2hgz
1Gb ram
jdk1.4.2
TC5.0.x ( have to double check the release number)

client:
gateway 450 laptop
1.4ghz centrino
1Gb ram
Jmeter
jdk1.4.2
keepalive is true

peter



PNG
1K
---
protocol | samples | average | median | 90% line | min | max | error%
| throughput | Kb/sec
---
HTTP Request50002.4158  0   10  0   231 0.00%   
1296.7/sec  1337.20
HTTP Request1   3.2866  0   10  0   490 0.00%   
1384.8/sec  1428.13
HTTP Request15000   3.56353 0   10  0   36260.00%   
1335.0/sec  1376.71
HTTP Request2   6.3406  0   20  0   120 0.00%   
1352.1/sec  1394.33
HTTP Request25000   8.27288 0   20  0   281 0.00%   
1309.1/sec  1350.02
HTTP Request3   4.6102  0   20  0   972 0.00%   
1259.2/sec  1298.53

5k

HTTP Request50002.9246  0   10  0   330 0.00%   
1044.5/sec  5314.28
HTTP Request1   4.4717  0   10  0   11620.00%   
1002.6/sec  5101.15
HTTP Request15000   6.7102  0   20  0   23730.00%   
968.9/sec   4929.49
HTTP Request2   4.83515 0   10  0   15030.00%   
950.1/sec   4834.10
HTTP Request25000   10.3578 0   30  0   19220.00%   
933.6/sec   4749.89
HTTP Request3   5.86573 0   20  0   441 0.00%   
939.4/sec   4779.46

10k

HTTP Request50004.1644  0   10  0   742 0.00%   
726.7/sec   7308.61
HTTP Request1   8.8132  10  20  0   181 0.00%   
763.5/sec   7678.00
HTTP Request15000   7.7906  0   20  0   381 0.00%   
718.0/sec   7221.14
HTTP Request2   10.8593 10  30  0   420 0.00%   
705.2/sec   7091.88
HTTP Request25000   32.2276 30  50  0   110 0.00%   
704.0/sec   7079.95
HTTP Request3   19.3326 10  50  0   16530.00%   
686.8/sec   6906.56

20k

HTTP Request50007.4202  0   10  0   400 0.00%   
491.9/sec   9936.63
HTTP Request1   14.1984 10  20  0   982 0.00%   
490.9/sec   9917.12
HTTP Request15000   17.698  10  40  0   531 0.00%   
472.4/sec   9542.36
HTTP Request2   22.9608 20  50  0   471 0.00%   
465.1/sec   9395.46
HTTP Request25000   32.8229 40  61  0   14320.00%   
461.3/sec   9318.23
HTTP Request3   50.4059 50  90  0   701 0.00%   
429.6/sec   8678.60

40k

HTTP Request50007.9292  0   10  0   11310.00%   
409.3/sec   8267.65
HTTP Request1   13.1405 10  21  0   10710.00%   
493.6/sec   9971.46
HTTP Request15000   18.5959 10  30  0   19230.00%   
517.0/sec   10444.56
HTTP Request2   32.9344 30  50  0   250 0.00%   
510.8/sec   10318.30
HTTP Request25000   38.1564 40  60  0   13620.00%   
493.4/sec   9966.44
HTTP Request3   52.4783 50  90  0   10010.00%   
463.5/sec   9363.52


HTML
1K
---
protocol | samples | average | median | 90% line | min | max | error%
| throughput | Kb/sec
---
HTTP Request50001.8508  0   10  0   170 0.00%   
1545.6/sec  1714.64
HTTP Request1   3.0936  0   10  0   321 0.00%   
1686.6/sec  1871.10
HTTP Request15000   3.3003  0   10  0   14220.00%   
1638.8/sec  1818.05
HTTP Request2   2.5934  0   10  0   110 0.00%   
1610.6/sec  1786.72
HTTP Request25000   3.99248 0   10  0   140 0.00%   
1537.1/sec  1705.26
HTTP Request3   4.12513 0   10  0   831 0.00%   
1473.5/sec  1634.72

5k

HTTP Request50003.1068  0   10  0 

some TC5 benchmark results for static file

[Peter Lin]

I've started a series of benchmarks to measure tomcat5 performance for
static files and compare it to apache2. Here are the results I have so
far. I thought others might find it interesting.

Server:
Redhat Fedora Core 1
AMD 2hgz
1Gb ram
jdk1.4.2
TC5.0.x ( have to double check the release number)

client:
gateway 450 laptop
1.4ghz centrino
1Gb ram
Jmeter
jdk1.4.2
keepalive is true

peter



PNG
1K
---
protocol | samples | average | median | 90% line | min | max | error%
| throughput | Kb/sec
---
HTTP Request50002.4158  0   10  0   231 0.00%   
1296.7/sec  1337.20
HTTP Request1   3.2866  0   10  0   490 0.00%   
1384.8/sec  1428.13
HTTP Request15000   3.56353 0   10  0   36260.00%   
1335.0/sec  1376.71
HTTP Request2   6.3406  0   20  0   120 0.00%   
1352.1/sec  1394.33
HTTP Request25000   8.27288 0   20  0   281 0.00%   
1309.1/sec  1350.02
HTTP Request3   4.6102  0   20  0   972 0.00%   
1259.2/sec  1298.53

5k

HTTP Request50002.9246  0   10  0   330 0.00%   
1044.5/sec  5314.28
HTTP Request1   4.4717  0   10  0   11620.00%   
1002.6/sec  5101.15
HTTP Request15000   6.7102  0   20  0   23730.00%   
968.9/sec   4929.49
HTTP Request2   4.83515 0   10  0   15030.00%   
950.1/sec   4834.10
HTTP Request25000   10.3578 0   30  0   19220.00%   
933.6/sec   4749.89
HTTP Request3   5.86573 0   20  0   441 0.00%   
939.4/sec   4779.46

10k

HTTP Request50004.1644  0   10  0   742 0.00%   
726.7/sec   7308.61
HTTP Request1   8.8132  10  20  0   181 0.00%   
763.5/sec   7678.00
HTTP Request15000   7.7906  0   20  0   381 0.00%   
718.0/sec   7221.14
HTTP Request2   10.8593 10  30  0   420 0.00%   
705.2/sec   7091.88
HTTP Request25000   32.2276 30  50  0   110 0.00%   
704.0/sec   7079.95
HTTP Request3   19.3326 10  50  0   16530.00%   
686.8/sec   6906.56

20k

HTTP Request50007.4202  0   10  0   400 0.00%   
491.9/sec   9936.63
HTTP Request1   14.1984 10  20  0   982 0.00%   
490.9/sec   9917.12
HTTP Request15000   17.698  10  40  0   531 0.00%   
472.4/sec   9542.36
HTTP Request2   22.9608 20  50  0   471 0.00%   
465.1/sec   9395.46
HTTP Request25000   32.8229 40  61  0   14320.00%   
461.3/sec   9318.23
HTTP Request3   50.4059 50  90  0   701 0.00%   
429.6/sec   8678.60

40k

HTTP Request50007.9292  0   10  0   11310.00%   
409.3/sec   8267.65
HTTP Request1   13.1405 10  21  0   10710.00%   
493.6/sec   9971.46
HTTP Request15000   18.5959 10  30  0   19230.00%   
517.0/sec   10444.56
HTTP Request2   32.9344 30  50  0   250 0.00%   
510.8/sec   10318.30
HTTP Request25000   38.1564 40  60  0   13620.00%   
493.4/sec   9966.44
HTTP Request3   52.4783 50  90  0   10010.00%   
463.5/sec   9363.52


HTML
1K
---
protocol | samples | average | median | 90% line | min | max | error%
| throughput | Kb/sec
---
HTTP Request50001.8508  0   10  0   170 0.00%   
1545.6/sec  1714.64
HTTP Request1   3.0936  0   10  0   321 0.00%   
1686.6/sec  1871.10
HTTP Request15000   3.3003  0   10  0   14220.00%   
1638.8/sec  1818.05
HTTP Request2   2.5934  0   10  0   110 0.00%   
1610.6/sec  1786.72
HTTP Request25000   3.99248 0   10  0   140 0.00%   
1537.1/sec  1705.26
HTTP Request3   4.12513 0   10  0   831 0.00%   
1473.5/sec  1634.72

5k

HTTP Request50003.1068  0   10  0   390 0.00%   
1078.3/sec  5713.64
HTTP Request1   5.1087  0   10  0   271 0.00%   
1154.5/sec  6117.33
HTTP Request15000   5.3688  0   10  0   15720.00%   
1096.6/sec  5810.54
HTTP Request2   4.46625 0   10  0   15020.00%   
1065.1/sec  5643.96
HTTP Request25000   8.9177

RE: unexpected timeout during benchmark

[Shapira, Yoav]

Hi,
How counter-intuitive...

Yoav Shapira
Millennium Research Informatics


>-Original Message-
>From: Henrik Rathje [mailto:[EMAIL PROTECTED]
>Sent: Thursday, September 09, 2004 12:41 PM
>To: Tomcat Users List
>Subject: Re: unexpected timeout during benchmark
>
>Hi,
>increasing the values of maxThreads and acceptCount decreases the
number of
>requests when the timeout occurs.
>
>On Thu, 9 Sep 2004 12:09:03 -0400
>"Shapira, Yoav" <[EMAIL PROTECTED]> wrote:
>
>>
>> Hi,
>> Maybe all the request processing threads are busy and the accept
queue
>> is full?  Check your Connector configuration and increase these
>> parameters if needed to allow your test to complete.
>>
>> Yoav Shapira
>> Millennium Research Informatics
>>
>>
>> >-Original Message-
>> >From: Henrik Rathje [mailto:[EMAIL PROTECTED]
>> >Sent: Thursday, September 09, 2004 12:05 PM
>> >To: [EMAIL PROTECTED]
>> >Subject: unexpected timeout during benchmark
>> >
>> >Hi,
>> >I want to measure the performance of a Servlet that I wrote. To have
a
>> >number to compare with i wrote a servlet which does only a single
>> println
>> >in its doget method.
>> >
>> >When i measure the throughput of this servlet with ab using this
>> arguments:
>> >ab -n 2 -c 10  http://localhost:8080/Messung1/TestServlet
>> >I receive:
>> >..
>> >Completed 14000 requests
>> >apr_poll: The timeout specified has expired (70007)
>> >Total of 15861 requests completed
>> >
>> >The jvm is defenitly not out of memory, (top says 50m of -Xmx170m
are
>> >used) no garbage collection is performed during the test. After some
>> >minites of waitung the servlet answers again (no tomcat restart
>> neccessary)
>> >What causes this timeout?
>> >(when running with half the amout of requests, the throughput is
>> 500/sec)
>> >Regards, Henrik
>> >
>>
>-
>> >To unsubscribe, e-mail: [EMAIL PROTECTED]
>> >For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>>
>>
>> This e-mail, including any attachments, is a confidential business
>communication, and may contain information that is confidential,
>proprietary and/or privileged.  This e-mail is intended only for the
>individual(s) to whom it is addressed, and may not be saved, copied,
>printed, disclosed or used by anyone else.  If you are not the(an)
intended
>recipient, please immediately delete this e-mail from your computer
system
>and notify the sender.  Thank you.
>>
>>
>> -
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
>-
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, e-mail: [EMAIL PROTECTED]




This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: unexpected timeout during benchmark

[Henrik Rathje]

Hi,
increasing the values of maxThreads and acceptCount decreases the number of requests 
when the timeout occurs.

On Thu, 9 Sep 2004 12:09:03 -0400
"Shapira, Yoav" <[EMAIL PROTECTED]> wrote:

> 
> Hi,
> Maybe all the request processing threads are busy and the accept queue
> is full?  Check your Connector configuration and increase these
> parameters if needed to allow your test to complete.
> 
> Yoav Shapira
> Millennium Research Informatics
> 
> 
> >-Original Message-
> >From: Henrik Rathje [mailto:[EMAIL PROTECTED]
> >Sent: Thursday, September 09, 2004 12:05 PM
> >To: [EMAIL PROTECTED]
> >Subject: unexpected timeout during benchmark
> >
> >Hi,
> >I want to measure the performance of a Servlet that I wrote. To have a
> >number to compare with i wrote a servlet which does only a single
> println
> >in its doget method.
> >
> >When i measure the throughput of this servlet with ab using this
> arguments:
> >ab -n 2 -c 10  http://localhost:8080/Messung1/TestServlet
> >I receive:
> >..
> >Completed 14000 requests
> >apr_poll: The timeout specified has expired (70007)
> >Total of 15861 requests completed
> >
> >The jvm is defenitly not out of memory, (top says 50m of -Xmx170m are
> >used) no garbage collection is performed during the test. After some
> >minites of waitung the servlet answers again (no tomcat restart
> neccessary)
> >What causes this timeout?
> >(when running with half the amout of requests, the throughput is
> 500/sec)
> >Regards, Henrik
> >
> >-
> >To unsubscribe, e-mail: [EMAIL PROTECTED]
> >For additional commands, e-mail: [EMAIL PROTECTED]
> 
> 
> 
> 
> This e-mail, including any attachments, is a confidential business communication, 
> and may contain information that is confidential, proprietary and/or privileged.  
> This e-mail is intended only for the individual(s) to whom it is addressed, and may 
> not be saved, copied, printed, disclosed or used by anyone else.  If you are not 
> the(an) intended recipient, please immediately delete this e-mail from your computer 
> system and notify the sender.  Thank you.
> 
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: unexpected timeout during benchmark

[Shapira, Yoav]

Hi,
Maybe all the request processing threads are busy and the accept queue
is full?  Check your Connector configuration and increase these
parameters if needed to allow your test to complete.

Yoav Shapira
Millennium Research Informatics


>-Original Message-
>From: Henrik Rathje [mailto:[EMAIL PROTECTED]
>Sent: Thursday, September 09, 2004 12:05 PM
>To: [EMAIL PROTECTED]
>Subject: unexpected timeout during benchmark
>
>Hi,
>I want to measure the performance of a Servlet that I wrote. To have a
>number to compare with i wrote a servlet which does only a single
println
>in its doget method.
>
>When i measure the throughput of this servlet with ab using this
arguments:
>ab -n 2 -c 10  http://localhost:8080/Messung1/TestServlet
>I receive:
>..
>Completed 14000 requests
>apr_poll: The timeout specified has expired (70007)
>Total of 15861 requests completed
>
>The jvm is defenitly not out of memory, (top says 50m of -Xmx170m are
>used) no garbage collection is performed during the test. After some
>minites of waitung the servlet answers again (no tomcat restart
neccessary)
>What causes this timeout?
>(when running with half the amout of requests, the throughput is
500/sec)
>Regards, Henrik
>
>-
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, e-mail: [EMAIL PROTECTED]




This e-mail, including any attachments, is a confidential business communication, and 
may contain information that is confidential, proprietary and/or privileged.  This 
e-mail is intended only for the individual(s) to whom it is addressed, and may not be 
saved, copied, printed, disclosed or used by anyone else.  If you are not the(an) 
intended recipient, please immediately delete this e-mail from your computer system 
and notify the sender.  Thank you.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

unexpected timeout during benchmark

[Henrik Rathje]

Hi,
I want to measure the performance of a Servlet that I wrote. To have a
number to compare with i wrote a servlet which does only a single println in its doget 
method.

When i measure the throughput of this servlet with ab using this arguments:
ab -n 2 -c 10  http://localhost:8080/Messung1/TestServlet
I receive:
..
Completed 14000 requests
apr_poll: The timeout specified has expired (70007)
Total of 15861 requests completed

The jvm is defenitly not out of memory, (top says 50m of -Xmx170m are
used) no garbage collection is performed during the test. After some
minites of waitung the servlet answers again (no tomcat restart neccessary)
What causes this timeout? 
(when running with half the amout of requests, the throughput is 500/sec)
Regards, Henrik

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Benchmark results for 4.1.12 vs 4.1.16

[Peter Lin]

 
I've done a fair amount of jsp and servlet development since 99 when JSP .9 spec 
wasn't implemented in most servlet containers. I doubt I can provide insight into 
improving reliability that tomcat developers haven't already thought of.
 
Most of the projects I've worked on the last three years have had some steep 
performance requirements like supporting a million users with 30% active daily. my 
last job the performance requirements were 500K to 1million pageviews a day. most of 
the reliability issues are from traffic spike and how to handle it gracefully. ie, you 
don't want the whole complex to die if traffic spikes. you also want to provide useful 
information to the user in the event the complex is overloaded.
 
probably the most important one is the server shouldn't die because the number of 
concurrent users jumped from 20/sec across the site to 300+/sec. In that reguard, 
tomcat 4 seems to do that. Well aside from developer/user errors :)
 
peter lin
 
 Remy Maucherat <[EMAIL PROTECTED]> wrote:Peter Lin wrote:
> today I took the day off, but tomorrow I plan on running more benchmarks. I ran into 
>one weird behavior when I ran the same jmeter test with 10 threads for 5K iterations. 
>Half way through tomcat stopped accepting connections, but was fine. If I wait two 
>minutes, it continue to accept requests.
> 
> That tells me tomcat is dramatically improved in stability. In the early days of 
>servlet containers, I used to run stress tests until the servers crashed. Back in 99 
>most of the servlet containers including websphere, jrun and tomcat would crash a 
>couple minutes after it stopped accepting connections. Now tomcat correctly denies 
>connections and performance degrades gracefully.


These days, I'm exclusively (well, almost) working on reliability and 
performance.

Performance is ok now (only remaining are a couple enhancements in 5.0), 
and is a lot easier to improve than reliability. If you have ideas for 
that, let me know.

Remy


--
To unsubscribe, e-mail: 
For additional commands, e-mail: 



-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now

RE: Benchmark results for 4.1.12 vs 4.1.16

[Shapira, Yoav]

Hi Peter,
Just wanted to say thank you for running these benchmarks.  They're
useful and insightful.

Yoav Shapira
Millennium ChemInformatics


>-Original Message-
>From: Peter Lin [mailto:[EMAIL PROTECTED]]
>Sent: Thursday, December 05, 2002 8:00 PM
>To: Tomcat Users List
>Subject: Re: Benchmark results for 4.1.12 vs 4.1.16
>
>
>today I took the day off, but tomorrow I plan on running more
benchmarks. I
>ran into one weird behavior when I ran the same jmeter test with 10
threads
>for 5K iterations. Half way through tomcat stopped accepting
connections,
>but was fine. If I wait two minutes, it continue to accept requests.
>
>That tells me tomcat is dramatically improved in stability. In the
early
>days of servlet containers, I used to run stress tests until the
servers
>crashed. Back in 99 most of the servlet containers including websphere,
>jrun and tomcat would crash a couple minutes after it stopped accepting
>connections. Now tomcat correctly denies connections and performance
>degrades gracefully.
>
>peter
>
>
> Remy Maucherat <[EMAIL PROTECTED]> wrote:Peter Lin wrote:
>> I forgot to mention the Jmeter settings I used.
>> All tests were with 1 thread for 5000 iterations. The first two image
>tests used images for tomcat. The third image test with 194.5k image
was a
>simple screen capture of my desktop.
>
>Thanks for the small benchmark. It is a bit odd that some tests are
>slightly worse, although it's well within the margin of error.
>If you have other tests (maybe with a higher load factor), send them.
>
>In 4.1.17, I'll provide a slightly more optimized default configuration
>for the connector (using disableUploadTimeout="true", and a bigger
>acceptCount, which may help avoid getting accept errors when under
load).
>
>Remy
>
>
>--
>To unsubscribe, e-mail:
>For additional commands, e-mail:
>
>
>
>-
>Do you Yahoo!?
>Yahoo! Mail Plus - Powerful. Affordable. Sign up now

--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

Re: Benchmark results for 4.1.12 vs 4.1.16

[Remy Maucherat]

Peter Lin wrote:

today I took the day off, but tomorrow I plan on running more benchmarks. I ran into one weird behavior when I ran the same jmeter test with 10 threads for 5K iterations. Half way through tomcat stopped accepting connections, but was fine. If I wait two minutes, it continue to accept requests.
 
That tells me tomcat is dramatically improved in stability. In the early days of servlet containers, I used to run stress tests until the servers crashed. Back in 99 most of the servlet containers including websphere, jrun and tomcat would crash a couple minutes after it stopped accepting connections. Now tomcat correctly denies connections and performance degrades gracefully.


These days, I'm exclusively (well, almost) working on reliability and 
performance.

Performance is ok now (only remaining are a couple enhancements in 5.0), 
and is a lot easier to improve than reliability. If you have ideas for 
that, let me know.

Remy


--
To unsubscribe, e-mail:   
For additional commands, e-mail: 

Re: Benchmark results for 4.1.12 vs 4.1.16

[Peter Lin]

today I took the day off, but tomorrow I plan on running more benchmarks. I ran into 
one weird behavior when I ran the same jmeter test with 10 threads for 5K iterations. 
Half way through tomcat stopped accepting connections, but was fine. If I wait two 
minutes, it continue to accept requests.
 
That tells me tomcat is dramatically improved in stability. In the early days of 
servlet containers, I used to run stress tests until the servers crashed. Back in 99 
most of the servlet containers including websphere, jrun and tomcat would crash a 
couple minutes after it stopped accepting connections. Now tomcat correctly denies 
connections and performance degrades gracefully.
 
peter
 
 
 Remy Maucherat <[EMAIL PROTECTED]> wrote:Peter Lin wrote:
> I forgot to mention the Jmeter settings I used.
> All tests were with 1 thread for 5000 iterations. The first two image tests used 
>images for tomcat. The third image test with 194.5k image was a simple screen capture 
>of my desktop.

Thanks for the small benchmark. It is a bit odd that some tests are 
slightly worse, although it's well within the margin of error.
If you have other tests (maybe with a higher load factor), send them.

In 4.1.17, I'll provide a slightly more optimized default configuration 
for the connector (using disableUploadTimeout="true", and a bigger 
acceptCount, which may help avoid getting accept errors when under load).

Remy


--
To unsubscribe, e-mail: 
For additional commands, e-mail: 



-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now

Re: Benchmark results for 4.1.12 vs 4.1.16

[Remy Maucherat]

Peter Lin wrote:

I forgot to mention the Jmeter settings I used.
All tests were with 1 thread for 5000 iterations. The first two image tests used images for tomcat. The third image test with 194.5k image was a simple screen capture of my desktop.


Thanks for the small benchmark. It is a bit odd that some tests are 
slightly worse, although it's well within the margin of error.
If you have other tests (maybe with a higher load factor), send them.

In 4.1.17, I'll provide a slightly more optimized default configuration 
for the connector (using disableUploadTimeout="true", and a bigger 
acceptCount, which may help avoid getting accept errors when under load).

Remy


--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

Re: Benchmark results for 4.1.12 vs 4.1.16

[Peter Lin]

I forgot to mention the Jmeter settings I used.
All tests were with 1 thread for 5000 iterations. The first two image tests used 
images for tomcat. The third image test with 194.5k image was a simple screen capture 
of my desktop.
 
peter
 
 Peter Lin <[EMAIL PROTECTED]> wrote:


I'm posting some early results of benchmarks I'm running on the different versions. 
Hopefully others will find the information useful.



The benchmarks were performed on two systems:



Server: 450mhz p3

512mb pc100 RAM

jdk 1.4.0

tomcat 4.1.12

tomcat 4.1.16



Client: 900mhz celeron

256mb pc133 RAM

Jmeter 1.7



Results 4.1.12



mean median stdev

---
2kb 4.1 0.0 15.7
8kb 5.8 0.0 20.5
16kb 7.3 10.0 23.9
1 8kb image 9.5 10.0 9.0
20 2kb image 76.2 71.0 32.2
194.5kb img 51.3 40.0 48.2




4.1.16 results

mean median stdev
---
2kb 3.6 0.0 9.5
8kb 4.7 0.0 13.6
16kb 6.0 10.0 15.8
1 8kb image 8.6 10.0 8.3
20 2kb image 77.5 71.0 38.3
1 194.5kb img 52.8 41.0 51.3





peter lin




-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now


-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now

Benchmark results for 4.1.12 vs 4.1.16

[Peter Lin]

 

I'm posting some early results of benchmarks I'm running on the different versions. 
Hopefully others will find the information useful.

 

The benchmarks were performed on two systems:

 

Server: 450mhz p3

512mb pc100 RAM

jdk 1.4.0

tomcat 4.1.12

tomcat 4.1.16

 

Client: 900mhz celeron

256mb pc133 RAM

Jmeter 1.7

 

Results 4.1.12



  mean median   stdev

---
2kb4.1   0.0 15.7
8kb5.8   0.0 20.5
16kb  7.3   10.0   23.9
1 8kb image9.5  10.09.0
20 2kb image  76.2 71.0   32.2
194.5kb img51.3 40.0   48.2


 

4.1.16 results

  mean   median   stdev
---
2kb 3.6   0.0 9.5
8kb 4.7   0.0 13.6
16kb6.0  10.015.8
1 8kb image 8.6  10.08.3
20 2kb image   77.5 71.038.3
1 194.5kb img  52.8 41.051.3

 

 

peter lin




-
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now

RE: benchmark

[Emerson]

what you mean by : cranked up the min and max processors to 2-3x the
defaults  ??

how you do that??

the tomcat was working with apache

>thanks for sharing your info!
>B
>
>-Original Message-
>From: Peter Lin [mailto:[EMAIL PROTECTED]]
>Sent: Friday, January 18, 2002 2:59 PM
>To: [EMAIL PROTECTED]
>Subject: benchmark
>
>
>
>Greetings everyone, a quick thanks to the tomcat
>developers for their hardwork.
>
>I just spent two week performing benchmarks on tomcat
>4.0.1 and I thought some people might like to know the
>results I got.
>
>My comparison was between Orion 1.5.2 and tomcat
>4.0.1. I used JMeter to perform the test hitting an
>application that gets XML data from another system
>over the internet. The data was about 1.2K in size and
>the parser used was xerces 2 and xalan.
>
>By no means is my test comprehensive, but it does show
>that tomcat 4.0.1 out performs orion 1.5.2 for 1-45
>clients. The application is a real app which I can't
>describe, but it performs two separate requests to an
>external system to get XML data, then transforms it
>into HTML. The system was a 900mhz p3 with 256m RAM,
>jdk 1.3.1.
>
>Tomcat vs Orion
>---
>1 client - 7% faster
>5 client - 2% faster
>15 clients - 9% faster
>45 clients - 20% faster
>
>The settings were the default setting for both. No
>optimization or tweaking was done. With the default
>settings for 90 and 135 clients, tomcat would fail.
>Once I cranked up the min and max processors to 2-3x
>the defaults, it was able to complete the tests.
>
>I hope others find this interesting.
>
>f00zbll
>
>__
>Do You Yahoo!?
>Send FREE video emails in Yahoo! Mail!
>http://promo.yahoo.com/videomail/
>
>--
>To unsubscribe:   <mailto:[EMAIL PROTECTED]>
>For additional commands: <mailto:[EMAIL PROTECTED]>
>Troubles with the list: <mailto:[EMAIL PROTECTED]>
>
>--
>To unsubscribe:   <mailto:[EMAIL PROTECTED]>
>For additional commands: <mailto:[EMAIL PROTECTED]>
>Troubles with the list: <mailto:[EMAIL PROTECTED]>
>
>
Emerson Cargnin
TRE-SC
Setor de Desenvolvimento 
Tel: (48) 251-3700 - Ramal 3134

--
To unsubscribe:   <mailto:[EMAIL PROTECTED]>
For additional commands: <mailto:[EMAIL PROTECTED]>
Troubles with the list: <mailto:[EMAIL PROTECTED]>

RE: benchmark

[Brian Adams]

thanks for sharing your info!
B

-Original Message-
From: Peter Lin [mailto:[EMAIL PROTECTED]]
Sent: Friday, January 18, 2002 2:59 PM
To: [EMAIL PROTECTED]
Subject: benchmark



Greetings everyone, a quick thanks to the tomcat
developers for their hardwork.

I just spent two week performing benchmarks on tomcat
4.0.1 and I thought some people might like to know the
results I got.

My comparison was between Orion 1.5.2 and tomcat
4.0.1. I used JMeter to perform the test hitting an
application that gets XML data from another system
over the internet. The data was about 1.2K in size and
the parser used was xerces 2 and xalan.

By no means is my test comprehensive, but it does show
that tomcat 4.0.1 out performs orion 1.5.2 for 1-45
clients. The application is a real app which I can't
describe, but it performs two separate requests to an
external system to get XML data, then transforms it
into HTML. The system was a 900mhz p3 with 256m RAM,
jdk 1.3.1.

Tomcat vs Orion
---
1 client - 7% faster
5 client - 2% faster
15 clients - 9% faster
45 clients - 20% faster

The settings were the default setting for both. No
optimization or tweaking was done. With the default
settings for 90 and 135 clients, tomcat would fail.
Once I cranked up the min and max processors to 2-3x
the defaults, it was able to complete the tests.

I hope others find this interesting.

f00zbll

__
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

--
To unsubscribe:   <mailto:[EMAIL PROTECTED]>
For additional commands: <mailto:[EMAIL PROTECTED]>
Troubles with the list: <mailto:[EMAIL PROTECTED]>

--
To unsubscribe:   <mailto:[EMAIL PROTECTED]>
For additional commands: <mailto:[EMAIL PROTECTED]>
Troubles with the list: <mailto:[EMAIL PROTECTED]>

benchmark

[Peter Lin]

Greetings everyone, a quick thanks to the tomcat
developers for their hardwork.

I just spent two week performing benchmarks on tomcat
4.0.1 and I thought some people might like to know the
results I got.

My comparison was between Orion 1.5.2 and tomcat
4.0.1. I used JMeter to perform the test hitting an
application that gets XML data from another system
over the internet. The data was about 1.2K in size and
the parser used was xerces 2 and xalan.

By no means is my test comprehensive, but it does show
that tomcat 4.0.1 out performs orion 1.5.2 for 1-45
clients. The application is a real app which I can't
describe, but it performs two separate requests to an
external system to get XML data, then transforms it
into HTML. The system was a 900mhz p3 with 256m RAM,
jdk 1.3.1.

Tomcat vs Orion
---
1 client - 7% faster
5 client - 2% faster
15 clients - 9% faster
45 clients - 20% faster

The settings were the default setting for both. No
optimization or tweaking was done. With the default
settings for 90 and 135 clients, tomcat would fail.
Once I cranked up the min and max processors to 2-3x
the defaults, it was able to complete the tests.

I hope others find this interesting.

f00zbll

__
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

--
To unsubscribe:   
For additional commands: 
Troubles with the list: 

Re: Benchmark

[David Cassidy]

Try

http://webperformanceinc.com

We've used it here and its really rather good.

You 'train' it by setting your browser to use it as a
proxy and then go through the bits of the site you want
to test. You login and it records the data you sent
Then you can get it to give your site alot of pain.
it will simulate alot of users doing standard user
type activity or you can get it just to blast your
site...

hope it helps

David


Laurent Michenaud wrote:

>
>
> Hi,
>
> I would like to bench my web server( apache/tomcat )
> What's the best ?
>
> I would like to indicate a list of urls( with somes params in it )
> and that it takes care of sessions.
>
> Michenaud Laurent
> - Adeuza -
> [ Développeur Web - Administrateur Réseau ]
>
> --
> To unsubscribe:   
> For additional commands: 
> Troubles with the list: 


--
To unsubscribe:   
For additional commands: 
Troubles with the list: 

Benchmark

[Laurent Michenaud]

Hi,

I would like to bench my web server( apache/tomcat )
What's the best ?

I would like to indicate a list of urls( with somes params in it )
and that it takes care of sessions.

Michenaud Laurent
- Adeuza -
[ Développeur Web - Administrateur Réseau ]


--
To unsubscribe:   
For additional commands: 
Troubles with the list: 

RE: tomcat benchmark

[Akbar Ahmed]

http://www.orionserver.com/benchmarks/benchmark.html

-Original Message-
From: shlomi sarfati [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 08, 2001 6:08 AM
To: [EMAIL PROTECTED]
Subject: tomcat benchmark 


does anyone have information about tomcat benchmark 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]

RE: tomcat benchmark

[Nael Mohammad]

I would like to see this as well. What is a good bench marking program to do
this? Something that can test the load it can handle would be nice and TPS.
Transactions per second. 

NAel

-Original Message-
From: shlomi sarfati [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 08, 2001 6:08 AM
To: [EMAIL PROTECTED]
Subject: tomcat benchmark 


does anyone have information about tomcat benchmark 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]

tomcat benchmark

[shlomi sarfati]

does anyone have information about tomcat benchmark 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, email: [EMAIL PROTECTED]