Vicenc Beltran Querol wrote:
Hi,
I've repeated the tests on the hybrid architecture using the AB.
You can find them attached to this mail. I've run the AB with several
concurrency levels, ranging from 20 to 10000. You can see all the
results in a plot.
-c 20 -k is basically the only thing I am interested in. This is not a
realistic test, it just measures the raw performance rather than the
scalability.
About your previous bench results: obviously the performance of the
regular HTTP connector is going to suck once the amount of connections
exceeds maxThreads. As most threaded servers, it scales by increasing
the amount of threads, and I believe it will perform relatively well (at
the expense of resources).
Running a test with ab (ab -k -c 20 -n 20000
http://host:8080/tomcat.gif) would take 30s, and would make comparisons
easy (basically, I actually know what the tests do ...), and will be an
actual measurement of throughput.
I've been studying the behavior of the AB and I've several doubts
about the significance of the results when trying to measure the throughtput
of a server. In my opinion, the AB is a great way to test the impact of
architectural modifications on the internal latency of the tomcat execution
pipeline, but not a deterministic way to compare the throughput of two servers.
In the following paragraphs I try to justify this idea (hope that in a
comprensible way) :)
Yes, you need to use ab to test either in localhost, or using a gigabit
network.
The first thing that makes me suspect about the reliability of the obtained
results
is that this benchmark produces a different workload intensity for each tested
server.
I mean that, given a number of concurrent clients simulated, the AB produces a
higher
number of requests as lower is the response time for the server (a new request
is issued
when the previous is completed). This behavior will always favour an
architecture
with lower internal latencies, even when it manages concurrency worse. This is
the case of
the tomcat multithreaded architecture. Other architectures, for instance the
Hybrid or any
other using non-blocking operations with readiness selectors, will always obtain
worse results for low loads (remember that the select operation introduces an internal
latency of 15-30ms since data is ready in a channel, with the purpose of
getting more
channels ready during that period).
When the simulated number of concurrent clients is increased (and especially
when the number of threads in the pool is lower than the number of emulated
clients), the multithreaded architecture starts suffering. You can check
the plots for the throughput, number of keep-alive requests, errors or connect
time to create your own opinion.
Well, that's precisely the reason why we never used non blocking IO in
the past :)
In conclusion, it must be taken into account that using this benchmark to
compare
the throughput of several architectural proposals can lead to wrong conclusions,
especially when WANs (instead of fast LANs) are used for the evaluation.
Yes.
This reasoning indicates that this test is more precise to compare the response
time between two server architectures than to evaluate its performance, because
the network latency (between the server and the client) can bias the obtained
results.
Finally, I miss a "think-time" as one of the configuration parameters of the
AB. It
reduces the "realism" of the test and makes not possible to test the performance
of the server in terms of "user sessions" instead of individual requests.
Again, I am not interested in a real world test here like you would do
on your server when putting it in production, and where you want to see
if your app reaches its performance targets, but a measurement of raw
performance.
PS: I'm very interested in your opinion (I mean the community) about my
reasoning
about the adequate use of the AB for throughput comparisons...
I don't see any results in your email, BTW.
Rémy
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
