On Apr 2, 2:01 pm, Paul Rogers <paul.rog...@shaw.ca> wrote:
> most load tools do direct http posts/gets/put requests. jmeter does this
> using a java library, to get the high levels of virtual users.
> Browsermob uses amazon cloud running ms windows 2003 server and a real
> browser.

Paul is absolutely right in this regard.  This is because the overhead
of rendering and running client side code means if you use anything
remotely resembling a 'real browser' it won't scale well, and you will
need a small fleet of systems to generate the load.

It's also un-necessary.  some vendors who don't support protocol level
load will try to justify their tool as being 'more realistic' which I
say is utter one-hundred percent hogwash.  The server has NO idea who
is sending those packets, and it can't tell IE or Firefox from a good
tool claiming to be IE or Firefox.. the 'realism' is all in the mind
of the salesman because the web server hasn't a clue.

>
> Ive think the best way of doing load tests is to use jmeter or other tool.
> You may be able to record a script via the jmeter proxy when watir is
> running. I can see a lot of advantages to this, but it takes a lot of hand
> editing to go and make the recorded jmeter script usable for more than one
> user.
>
> Ive found load testing to be a particulary difficult thing todo. Mostly
> because of the question "what are we trying to do here?" - find out what bit
> breaks first? Find out how many users we can support before we need a new
> bit of hardware? Find out what the slowest operastion is? Find out how to
> make the slowest operation better? There is a book that Scott Barber was a
> coauthor of, that is available as a free pdf that is good reading.

I agree 100% Paul.  load testing and perf testing is some of the more
'scientific' testing you can do, it requires close attention to
detail, tight control over variables, tests that are highly
reproducable, and as you point out you have to start out by asking the
right question.  A loadtest is basically an experiment, and like any
experiment it's only going to be as successful as the hypothesis it's
trying to prove or disprove.  If you don't know the question you are
trying to ask, you're not likely to design loadtest that will give you
results you can use.

As for myself, I've used both loadrunner (really good but really
expensive) and Microsoft's Visual Studio Team System2008 (both the
"test edition" and "team suite" versions have a HTTP level web-test
and loadtest capability)  I found in my work that it scales well and
blows loadrunner away in terms of bang for the buck.  I've used it
with just a single machine as the load generator (also running the
controller as wel) and brought both webservers and db servers to their
virtual knees and had they crying uncle!  There's also an excellent
MSDN forum with some of the main pm's etc providing advice and
support, and a small collection of videos and such.

The downside is that it does all the SUT monitoring via WMI
(Microsoft's implementation of WBEM) which works fabulous on windows
servers, but because the *nix community never implemented WBEM (a
multi vendor DMTF standard) there's no performance monitoring daemon
that allows you to connect to the machine using WBEM, and hence you
can't monitor the performance of non windows servers, and without good
(really good) data on how the systems are being affected by the load,
corrilated with what the scripts are doing, the loadtests are of much
less value.

Sorry but this is a bit of a soapbox for me. it's really ironic in a
way that the "closed' windows OS, because MS implemented an open
standard is more accessable in terms of remote monitoring and
rathering of system information (out of the box without having to
installl anything, and merely configuring some accounts that are
allowed to access the data) than the supposedly 'open' os's which
never adopted that standdard and are thus a relative black hole in
terms of being able to remotely monitor performance counters

--Chuck
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Watir General" group.
To post to this group, send email to watir-general@googlegroups.com
Before posting, please read the following guidelines: 
http://wiki.openqa.org/display/WTR/Support
To unsubscribe from this group, send email to 
watir-general-unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/watir-general
-~----------~----~----~----~------~----~------~--~---

Reply via email to