More performance tests from the OFBiz guys - this time they're much more interesting for Orion users!
The most interesting stat sections are the sheer performance (where Orion absolutely wallops the competition) and price/performance. (quoting - sheer performance) Here they are, in order from fastest to slowest: Orion 1.5.3: 0.045s (45 milliseconds) - 444 pages/sec - 1.6 million/hour Resin 2.0.4: 0.065s (65 milliseconds) - 307 pages/sec - 1.1 million/hour Weblogic 6.1: 1.15s (1,150 milliseconds) - 17.4 pages/sec - 62,640/hour Tomcat 4.0.1: 3.3s (3,300 milliseconds) - 6 pages/sec - 21,600/hour (quoting - price/performance including $850 for a decent server) Resin 2.0.4: 1.1million/(500 + 850) = 814 Orion 1.5.3: 1.6million/(1500 + 850) = 680 Tomcat 4.0.1: 21,600/(0 + 850) = 25.4 Weblogic 6.1: 62,640/(10,000 + 850) = 5.77 (please remember here that Resin 2.0.4 is a servlet/jsp container, whereas Orion is a full J2EE server so they have slightly different feature sets despite comparing very favourably on price/performance). Hope this is interesting to others! Cheers, Mike PS Next time someone tells you that JBoss+Tomcat can outperform Orion, you have some excellent data to prove them wrong PPS Wise man once say anyone who trusts benchmark tests absolutely is a fool - test yourself, on your own apps! ------ Forwarded Message From: "David E. Jones" <[EMAIL PROTECTED]> Organization: Open For Business Reply-To: [EMAIL PROTECTED] Date: Thu, 7 Feb 2002 17:09:51 -0700 To: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: [Ofbiz-devel] OFBiz Performance on various app servers after optimizing OFBiz Last night I sent a message to the ofbiz-devel list reporting the results of some performance enhancements in OFBiz from optimizing certain little pieces of code. I did some more tests today on the speed of the new improved OFBiz on various app servers. The following numbers are average server response times with 20 threads continuously hitting the server (ie when each thread gets a response it immediately sends another hit, keeping about 20 requests in the queue at all times). The test was done with JMeter on one computer and the app server & OFBiz on my PIII 1Ghz laptop. Running Linux (kernel 2.4.10, Suse 7.3) and Sun's JDK 1.4 for all but Weblogic, which requires JDK 1.3, and yes 1.4 is slightly (like 5%) faster. The page hit was ecommerce/control/main with the default catalog. It had two products in the Featured Products category and two top level categories displayed on the side. This isn't a very realistic test for large catalogs, but is a medium sized page to test the app server container preformance. Here they are, in order from fastest to slowest: Orion 1.5.3: 0.045s (45 milliseconds) - 444 pages/sec - 1.6 million/hour Resin 2.0.4: 0.065s (65 milliseconds) - 307 pages/sec - 1.1 million/hour Weblogic 6.1: 1.15s (1,150 milliseconds) - 17.4 pages/sec - 62,640/hour Tomcat 4.0.1: 3.3s (3,300 milliseconds) - 6 pages/sec - 21,600/hour The pages per second counts are calculated as follows: (1/avg. time)*20, because of the 20 continuous threads hitting it. As you can see Orion & Resin perform very similarly but Weblogic and Tomcat are left in the dust. Before optimizing OFBiz Weblogic was the fastest, coming in about twice as fast as Orion and Resin. I know from past (good?) experience with Weblogic that they have a lot of tuning parameters, so chances are you can increase the thread pool size or something to get it to go faster. I reduced the number of threads hitting it and it did somewhat better, but never came close to the speed of Orion or Resin, it's webapp container must be just plain heavier. For the $10,000 per CPU range, I think I'll pass, even though I'm sure it can do better than what it was in this test (I don't think it'll ever touch Orion or Resin for webapp speed). Tomcat was by far the slowest. It seemed to run all right for a little while, and then have little periods of slowness where page times jumped up from an average of about 2 seconds to an average of about 5 seconds. Over the minute it levelled out to about 3.3 seconds as listed above. So, which to choose? If you need EJB and other features that Orion provides and want to pay more for them ($1500 per server), then go for it. It is a little bit difficult to configure (or maybe it's just me?), but it runs REALLY well. If you have a smaller budget and are fine with Tyrex (open source from Exolab) as your DB connection pool and TX monitor, then go for Resin. OFBiz runs great on it, and it's only $500 per server. On price/performance Resin wins (pages per second/dollars): Resin: 1.1million/500 = 2200 Orion: 1.6million/1500 = 1066 Weblogic: 62,640/10,000 = 6.264 Tomcat: 21,600/0 = N/A With that sort of calculation you could argue that Tomcat really does win, beacuse it costs nothing so the (pages per second/dollars) would be infinite, effectively. But lets factor in the cost of a server. A cheap PIII 1Ghz supported server from Dell (PowerEdge 500SC) that is similar to my laptop with 512Mb RAM, plain IDE hard disk (doesn't matter much on an app server) with 3yrs of next business day support is $850. So, lets recalculate with that in the mix: Resin 2.0.4: 1.1million/(500 + 850) = 814 Orion 1.5.3: 1.6million/(1500 + 850) = 680 Tomcat 4.0.1: 21,600/(0 + 850) = 25.4 Weblogic 6.1: 62,640/(10,000 + 850) = 5.77 So, from this calculation we can see that for the price, you will get the best volume from Resin. When you consider the hardware price in with everything else, Resin will give you 32 times more bang for the buck, and Orion will give you VERY close to that, in spite of being more expensive: 26 times more bang for the buck. Pretty interesting, huh? If you would like to do some performance tests and need more detail, let me know, but you should have most of the detail you need here. Do post something back if you get very different results, or if you can tweak Weblogic to get better numbers (especially if they get better than Orion or Resin, because that would surprise me). Later, -David Jones _______________________________________________ Ofbiz-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/ofbiz-devel ------ End of Forwarded Message
