Hi Derek,
It's great that you started to nail down the attachment performance
issues.. I've also started to do some refactorings to the Attachments
handling classes.. I still have some test cases failing., which means
it'll take some more time to get the changes in..
Let's crack this together :)... I'll also do some profiling to get more clues..
~Thilina
On 3/7/07, Derek Clayton <[EMAIL PROTECTED]> wrote:
Hi Davanum
Unfortunately, the server is in .NET so I would only be able to provide the
client and some other sample data which alone might not be useful.
However, I did some research in to the Axis2 and Axiom source using a client
that was trying to read a ~1.8M binary attachment (XOP/MTOM). It was taking
just over 3 seconds to read in and save the file. I finally tracked down
that 98% of the time involved was occuring in the Class:
org.apache.axiom.attachments.MIMEBodyPartInputStream specifically in the
read() method at the line:
// read the next value from stream
int value = inStream.read();
The inStream is a PushbackInputStream. On a simple isolated test (i.e.
plain old java reading from the file system) I tested reading in the file
using a BufferedInputStream vs a PushbackInputStream. It took 31 ms for the
BufferedInputStream and a whopping 1047 ms for the PushbackInputStream. (I
know there are reasons for the use of PushbackInputStream)
I thought I had found the reason why it was slow and even in the Axiom
source just before it creates the PushbackInputStream it states in the Class
org.apache.axiom.attachments.Attachments:
// do we need to wrap InputStream from a BufferedInputStream before
// wrapping from PushbackStream
So I changed the source to first wrap in a BufferedInputStream however there
was no change in performance. The immediate Underlying InputStream is a
org.apache.commons.httpclient.AutoCloseInputStream and whatever it might
wrap (etc) I'm not sure.
So basically I'm still not sure about what's going on. PushbackInputStream
is definitely a lot slower because it does not do any buffered reads.
However, I don't know why the performance wasn't improved when I wrapped it
in a BufferedInputStream. I was unable in my investigative time to find out
any other InputStreams that might have ultimately been wrapped by the
PushbackInputStream.
Derek
>From: "Davanum Srinivas" <[EMAIL PROTECTED]>
>Reply-To: [EMAIL PROTECTED]
>To: axis-user@ws.apache.org
>Subject: Re: MOTM and Encoding/Decoding
>Date: Thu, 1 Mar 2007 14:07:25 -0500
>
>Derek,
>
>Could you please log a JIRA bug and upload the sample code? Let's try
>to create the scenario on our boxes...
>
>thanks,
>dims
>
>On 3/1/07, Derek Clayton <[EMAIL PROTECTED]> wrote:
>>Thank you for the very fast response.
>>
>> >If possible please post us some numbers of the time comparison. Make
>> >sure to avoid the System.out part when doing the comparison (this
>> >encoding takes time :( )...
>>
>>First let me explain the setup. The two machines are identical
>>hardware...Pentium 4 2.8GHz with 1Gig memory. Software differs since
>>these
>>are two different developer machines.
>>
>>Machine 1 acts as the client and is written in Java using Axis2. It sends
>>an Excel file to Machine 2. When it receives the XML file back (see next
>>paragraph) is simply saves it as a file.
>>
>>Machine 2 acts as the server and is using .NET. It receives an Excel file
>>as binary, saves that file, uses an ActiveX control to read in the file
>>and
>>parse it to generate an XML representation of the data. It then sends
>>that
>>XML back to the Machine 1 in the SOAP response.
>>
>>I haven't written an isolated test but this setup would favor Java/Axis2
>>anyway since Machine 2 is having to do actual work in addition to reading
>>the binary file.
>>
>>For smaller Excel files the times are quite reasonable.
>>
>>Test 1
>>--------------------------
>>For a 302K send and a 922K response.
>>
>>Time to send and receive response: 1.6 secs.
>>Time to read response: 6 secs.
>>
>>It is taking about 3 times longer which is reasonable since the file size
>>is
>>3x as large even though the server is doing a lot more work than the
>>client.
>>
>>However, as the files get large the performance begins to break down.
>>
>>Test 2
>>--------------------------
>>For a 1.4M send and a 5.4M response.
>>
>>Time to send and receive response: 3 secs.
>>Time to read response: 37 secs.
>>
>>37 seconds to read the 5.4M binary file seems like a long time. As well
>>you
>>can see that the server processed a larger file (1.4M) in half the time as
>>did the client in Test 1. For even larger files the difference becomes
>>greater.
>>
>>Test 3
>>--------------------------
>>For a 8.5M and a 32M response.
>>
>>Time to send and receive response: 18 secs.
>>Time to read response: 220 secs.
>>
>>Here you can see the .NET is reading the 8.5M (along with parsing it and
>>generating XML) in 18 secs compared to where Java/Axis2 took 37 secs to
>>simply read and save a 5.4M file in Test 2.
>>
>>I know this is an informal test case. But is 37 seconds to read a 5.4M
>>binary attachment with no decoding similar to the experience of others?
>>
>>Thanks!
>>
>>Derek
>>
>>_________________________________________________________________
>>Don't waste time standing in line—try shopping online. Visit Sympatico /
>>MSN
>>Shopping today! http://shopping.sympatico.msn.ca
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: [EMAIL PROTECTED]
>>For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>
>
>
>--
>Davanum Srinivas :: http://wso2.org/ :: Oxygen for Web Services Developers
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, e-mail: [EMAIL PROTECTED]
>
_________________________________________________________________
Your Space. Your Friends. Your Stories. Share your world with Windows Live
Spaces. http://spaces.live.com/?mkt=en-ca
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Thilina Gunarathne
WSO2, Inc.; http://www.wso2.com/
Home page: http://webservices.apache.org/~thilina/
Blog: http://thilinag.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
