For me, I want a comprehensive suite of tests to run. I want to measure the response times between a client and the server, the time a server gets a request until the time it goes back, the number of threads running, the number of simultaneous requests going at one time. I want to see reports on the scaling capabilities of JBoss. One node, two nodes with round robin, two nodes with first available, three nodes, two partitions, etc. I want to see how close to linear scaling occurs when adding more nodes, more partitions, etc. I want to see for myself that my idea of a good cluster architecture is indeed good, or maybe another way is better. I want to do these tests for stateless ejb and statefull ejb, and while my specific needs to not entail a web application, I would also like to see the performance with things like a client sending web service requests, which in turn hit the ejb server, and so forth.
Ideally, it would be very strong for JBoss to have these and many more test results available. Realistically, we both know that no one app is going to certify JBoss, every app will no doubt have different characteristics, and thus results. But it would be great to provide a client/j2ee type of tool that can be used/deployed in any J2ee server to see the various results. Running Orion, Jonas, JBoss and Primatti side by side (one at a time of course) on the same hardware with the same test data and same setup would be very nice, to see which one supports better load, better clustering, better scaling, and so forth. These are the types of results I am after. --- Jon Barnett <[EMAIL PROTECTED]> wrote: > Kevin, Bela; > > First thanks for the offer of help, Bela. Being > downunder, sometimes you > feel a bit isolated - where is the Australian JBoss > user community? ;) > Anyway, I'm not sure where things are headed with > testing but help is > always appreciated. > > The testing question is a big one. It depends on > what you are trying to > achieve. > > If it is steady state operation, I would say that a > cluster of JMeter > clients can generate a reasonable load that you can > tune. Based on the > threading issues discovered here, I wouldn't have > the client on the same > machine under test if you are using Linux. There > would be too many > questions on client load versus server load. One of > the nice things about > hindsight is that the free-running client load > testing told us something > about what to expect when looking at the > JBoss-client results. So for > completeness, you would probably want a calibration > sample of your test > rig with the minimal JBoss interaction as we had > done in the JBoss-client > test. Then you can look at responsiveness under a > certain load > characteristic - for example 50 requests per second > what is the average > response time, what are the maxima and minima. Dial > it up and check again. > As long as the test rig can generate the requested > load level (calibration > will tell you when things might start to deform in > the test generator), > you will be reasonably certain about the performance > of the system under > test. > > If you are just stress testing the system to check > raw throughput, then > you can adopt the load and transmit operation we > employed for this test. > Except we were hoping for less parallel thread > coupling effects. You were > able to see from these results for example, that the > name server and the > EJB invocation were not unduly stressed with an 8 > client-load continually > making requests for an extended period (for the IBM > SDK). The predominant > response characteristic was that of the client > system. > > We haven't looked at your particular idea but any > measurement is better > than no measurement at all. Besides, I think there > needs to be something > beyond the ECperf-type tests. They tell you how the > big picture is > supposed to work but don't help you work out where > things fall apart. > > I would suggest that measurement data if possible, > is not transmitted from > the server during testing (unless you can guarantee > its affect would be > minimal to the testing results). > > I think the best thing is to flesh out what you want > to come away with > from the testing. I'm sure there are others here in > the forum with good > test and measurement experience who can contribute > to the cause. An > example of things to think about - do you want to > test solely EJB > performance? Do you want to couple it with a DB? How > do you separate DB > from Server issues? > > Some nice things to see on the cluster testing is > the impact of the load > distribution algorithm. Is there a step-over point? > How would you test it? > I think it is a matter of asking what you want to > observe and then > determining a test scenario that would allow you to > see it. > > Now if some Australian company can donate a Sun > multi-CPU Blade server for > testing, I could check the performance of the Java > thread implementation > on a Sun system. :) > > Anyway, that's a starter for discussions. I know I > haven't directly > answered your question about how to proceed but I am > just trying to get a > feeling about the exact things you want to measure. > > Best regards, > > JonB > > > ATTACHMENT part 2 application/x-pkcs7-signature name=smime.p7s __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ------------------------------------------------------- This SF.Net email sponsored by: Free pre-built ASP.NET sites including Data Reports, E-commerce, Portals, and Forums are available now. Download today and enter to win an XBOX or Visual Studio .NET. http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01 _______________________________________________ JBoss-user mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-user
