Well, here's exactly why I hate benchmarking: it's never consistent! :)
This time, everything compiled with JDK 1.6 (it's what I have installed
along side 1.4.2) and all used 5000 samples (updated test plan to always
do 5000 samples)...
S1:
685 average, 142.3/sec throughput, 20.89 KB/sec
So, an improvement there, maybe because of the JDK, maybe not.
S2 w/OGNL 2.6:
1054 average, 87.3/sec throughput, 12.87 KB/sec
That's better than the last run, but for no apparent reason!
S2 W/OGNL 2.7:
1073 average, 85.8/sec throughput, 12.65 KB/sec
Wuh?!? Ok, I think we can most likely dismiss the difference as within
a statistical margin of difference, which means either (a) these tests
are just flat-out flawed somehow, (b) the OGNL bump, at least as far as
just a straight drop-in, makes no real difference, or (c) OGNL isn't
being used to enough of an extent in this test to notice a difference
(I'm frankly betting on that one).
FYI, I've created JIRA ticket WW-2040... that way if others want to
extend the apps, they can attach an updated version to the ticket so
everyone can share.
Frank
cilquirm wrote:
Frank, would you care to give the same tests a shot with ognl 2.7 and
javassist in the mix.
Although none of this is purely scientific, at least evaluations on that
regard give us some level of subjective information.
The ognl 2.7 and javassist jar are available via the tapestry-4.2-libs
download :
http://tapestry.apache.org/download.html
Frank W. Zammetti wrote:
Dunno if this might help, but:
http://www.omnytex.com/struts_benchmarking.zip
In it you'll find two applications, one for S1 (1.3.8) and one for S2
(2.0.8)... they are both (I think!) pretty much equivalent, and about as
simplistic as you can get. Also included is a JMeter test plan to run
against them (just disable one or the other thread group, wouldn't want
to test them both at the same time!).
Just ran a quick-and-dirty comparison of the two using the test plan...
I ran 100 users with no ramp-up... local Tomcat instance (6.0.13)... the
one difference is that the S1 version was compiled with JDK 1.4.2, and
the S2 version with 1.6.0, so there's at least one potentially big
variance right up front... here's what I saw:
S1 results:
4256 samples, 913 average, 108.6/sec throughput, 16.01 KB/sec
S2 results:
4165 samples, 1974 average, 50.0/sec throughput, 7.38 KB/sec
I'm not claiming this to be the perfect test, nor do I believe there's
not some flaws in there (benchmarking is always a tough thing to get
quite right, especially trying to do a comparison like this)... but,
unless someone can point out some obvious mistakes I made, the numbers
don't lie: S2 *looks*, *on the surface* at least, to be inherently twice
as slow as S1.
I'm not trying to make any sensational claims here, and again, I may
have totally blown it in the first place (I did throw this together in
about 30 minutes after all), but if we can use this as a basis going
forward, maybe build it up as a more expansive, realistic and solid
benchmarking suite, then it's all good in the end.
Anyway, it's there, if anyone's interested.
Frank
--
Frank W. Zammetti
Founder and Chief Software Architect
Omnytex Technologies
http://www.omnytex.com
AIM/Yahoo: fzammetti
MSN: [EMAIL PROTECTED]
Author of "Practical Ajax Projects With Java Technology"
(2006, Apress, ISBN 1-59059-695-1)
and "JavaScript, DOM Scripting and Ajax Projects"
(2007, Apress, ISBN 1-59059-816-4)
Java Web Parts - http://javawebparts.sourceforge.net
Supplying the wheel, so you don't have to reinvent it!
Ing. Andrea Vettori wrote:
Il giorno 12/lug/07, alle ore 16:31, Guillaume Carré ha scritto:
2007/7/12, Ing. Andrea Vettori <[EMAIL PROTECTED]>:
Compared to nothing... they are pure numbers. They are simply just
good enought (to me).
If we don't have this in mind we should use assember for everything :)
what I meant was: maybe it could be a good idea to redevelop your
screens with, say struts 1 for example, and compare the results
mmmm I can't do that... simply don't have the time... :)
In the high load test, after 10 seconds you have about 90 users (the
other 10 should have finished). Having a response time of 2,5 seconds
for a db search and result display under such load seems very good to
me.
Don't you ?
it depends :-)
"2.5s" doesn't say much to me, I would need to know how much time is
consumed in your DB requests, how much time is consumed in your
service layer, etc etc.
Is it 90 users really active at the same time, meaning using 90
threads on the server?if it is, do you have at least 90 connections in
your pool?
or did you put think times in your tests?
No think time...
I have 250 threads but I have a limit of 50 connections on my pool. I'll
try to raise
the number of maximum connection to see if the MP3 list test gets better.
However I think that struts alone is performing well for my app; don't
know if it's because
i'm using only few OGNL expressions on my jsp pages.
--
Ing. Andrea Vettori
Consulente per l'Information Technology
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--No virus found in this incoming message.
Checked by AVG Free Edition.Version: 7.5.476 / Virus Database:
269.10.4/897 - Release Date: 7/11/2007 9:57 PM
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
--
Frank W. Zammetti
Founder and Chief Software Architect
Omnytex Technologies
http://www.omnytex.com
AIM/Yahoo: fzammetti
MSN: [EMAIL PROTECTED]
Author of "Practical Ajax Projects With Java Technology"
(2006, Apress, ISBN 1-59059-695-1)
and "JavaScript, DOM Scripting and Ajax Projects"
(2007, Apress, ISBN 1-59059-816-4)
Java Web Parts - http://javawebparts.sourceforge.net
Supplying the wheel, so you don't have to reinvent it!
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
