Re: [service-orientated-architecture] Performance... how important is it?

[Steve Jones]

Interestingly we did a smilar test (about 2 years ago, and XML has come on since then) and we found that RMI v WS was a very close call indeed, over a bandwidth limited connection when using compression on the XML documents (Apache module) the SOAP actually outperformed the RMI.  I completely agree that 2.7 seconds would be ridiculous, it would be great to understand which toolkit is delivering that poor performance as using AXIS and AXIS 2 I'm not seeing anything close to that sort of overhead.

This http://grid.cs.binghamton.edu/projects/soap_bench/img/2.html is something I came across which has AXIS performing at sub 0.014 for 25,000 elements in an array, there are a bunch of Sun v MS bitch fights ( http://java.sun.com/performance/reference/whitepapers/WS_Test-1_0.pdf for the Sun one, the MS one was at middleware research which is no more) and AXIS 2 has some figures out ( http://www.wso2.net/2006/05/axis2_performance_testing_round_1) as well.  All of these are well inside the 500 objects in 2 seconds.  The SOAP benchmark paper is at http://grid.cs.binghamton.edu/projects/publications/soapBench-SC05/soapBench-SC05.pdf.

It does seem odd how little effort appears to have gone into SOAP performance analysis.

On 28/10/06, Donahue Bryon <[EMAIL PROTECTED]> wrote:

Yes, performance is critical and it severely hinders
what we can do with WS; it is not that our performance
requirements are severe but rather that the WS
toolkits perform poorly. The issue is that SOAP/Java
marshalling is pathetic, just pathetic. Until this is
fixed, WS will struggle in low-latency or large data
set enterprise environments.

We analyze performance in terms of latency. Yes,
transaction latency if often dominated by I/O but in
the case of WS, middleware latency may dominate.

We have a simple test that consists of an echo Array
operation that operates on an array of structures. The
structure consists of 5 elements and the array is 25
binary bytes. We use this test with a number of tools
and the results are consistent across the board.

Below are results, in milliseconds, with a CORBA
toolkit and a commercial JAX-RPC product. The test
environment is not important but relative behavior is.

Array Size 10 500
-----------------------------------
JAX-RPC 40 2715
CORBA 2 25

Consider that the middleware latency alone for echoing
a 500 element array is 2.7 seconds for WS vs 2
milliseconds for CORBA. Few transactions can tolerate
that kind of middleware latency.

We stop our test suite at 500 array elements because
the latency of WS toolkits becomes absurd or they
start throwing memory exceptions.

Bryon

--- Steve Jones <[EMAIL PROTECTED]> wrote:

> While I bang on about technology not being important
> with SOA, I'd
> like to kick a thread off around one of my
> perceptions.
>
> Anne said at the SOA event in Belgium this week that
> she'd recommend
> XML gateways for performance reasons.
>
> My view (and the experience I've had tends to back
> this up) is that
> XML performance is rarely an issue. The cost of
> marshaling WS against
> RMI is minimal in most cases, and when using a StAX
> parser for
> fragments of a large document then the WS approach
> will be much
> quicker than RMI. And generally with the current
> hardware performance
> characteristics (My laptop is a 2-way 2GHz box) the
> challenge isn't on
> the computational side for performance around XML
> processing but still
> in elements like databases, transaction managers and
> of course the
> actual IO (like disk) and bandwidth elements.
>
> Note here I'm talking about the 95%+ of applications
> which measure
> end-to-end performance in the second or 1/10 second
> margins, not at
> the micro-second level.
>
> So while XML might be one of the most inefficient
> transports ever
> dreamed of. Is it really an area where most
> organisations should
> worry about performance given the current hardware
> characteristics?
>
> What is the experience out there? Is XML
> performance really that bad
> in most software stacks that dedicated solutions are
> required? Stats
> and data most appreciated.
>
> Steve
>
>

__________________________________________________________
Access over 1 million songs - Yahoo! Music Unlimited
(http://music.yahoo.com/unlimited)

__._,_.___

SPONSORED LINKS

Computer software	Computer security software	Computer software program
Computer fax software	Computer virus software

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___