On 07/25/2011 02:36 PM, Oliver Lloyd wrote:
Hi Eric, what you are talking about is running a capacity test - what level
of load can my system sustain before degradation occurs, right?
No the test is much more simple, its simply what is the maximum number
of HTTP transaction that can be processed when the application is 80%
cached and there are edit events occurring at a rate consistent to what
is seen in the field (about 4 a second).
This number is reasonable simple to gather for a single server as I only
need to use the jmeter.sh locally in which case I can get just over 135
transactions a second. However I need to test it in a cluster. The
problem occurs in that as soon as I try to use client-server the
performance (on the same single server) that is measured drops to only
about 40 hits per second. This appears to be because jmeter is not
actually driving load properly, the actual response times from the
server are identical to that of using the local setup at about the same
load level (done by reducing the number of threads run), and the load is
about the same, the problem is jmeter does not appear to be sending as
many requests as it should as quickly as it should, AKA client-server is
behaving very different that the local version.
In this case you still need to define a specific load and control the rate
of requests you are sending. Sure, once you have a defind request rate there
is nothing to stop you gradually ramping it up to find the break point -
this is a standard test. But if you don't set pacing and simply let the test
run as fast it can then you tend to get into problems. I'm not saying
there's no benefit and it won't work, just that it is *better* to run
controlled tests.
One example why: Without pacing each request will begin as soon as the last
one finished, once things start to slow down then automatically the rate of
requests will reduce - the thread will not be able to make as many requests
and the load will reduce - this causes odd/confusing behavior. With pacing
you keep a constant load even when response times increase.
I have pacing setup via "uniform random Timer", for edits its
1200ms+-200ms and for reads its 150ms+-100ms
Another: without control you will find that faster machines will run your
test faster - it will mean that is it not a repeatable test. Any tester
worth his salt has the concept of repeatability deeply ingrained in his
psyche - you must have repeatability. Imagine if you run a test on your
boxes and get limit Y and then your client runs it on, some other different
boxes, and gets result X. How is this useful? If you control the rate this
will not happen.
All load generators and tested servers in this case are identical so
this should not be an issue.
Another: when you hit a limit, how do you know this is a problem or not?
Every system (every single one) has limits and as long as you keep ramping
up you will always keep hitting limits - so how do you know when to stop?
As I said above this is a very simple test of maximum HTTP requests that
can be handled, so in this case I do not care about what is the
bottleneck as much as what is the HTTP requests a second processed. Now
obviously I run other tests to remove potential bottlenecks and tune the
systems, but for the report I must produce all I need are HTTP Requests
a second, my problem is in determining why client-server jmeter behaves
very differently that local jmeter.
You have to set yourself targets/objectives, this is golden rule numero uno.
Even with stress testing you need to control the rate/distribution of the
load.
But hey, this is just my opinion, but it's based on experience, that's why
you come to forums like this, no?
--
View this message in context:
http://jmeter.512774.n5.nabble.com/Jmeter-Performance-using-jmeter-server-VS-running-in-the-local-instance-tp4631144p4631900.html
Sent from the JMeter - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
Thanks,
ERIC
---------------------------------------------------------------------
To unsubscribe, e-mail: jmeter-user-unsubscr...@jakarta.apache.org
For additional commands, e-mail: jmeter-user-h...@jakarta.apache.org
