Re: MTOM Attachments and PushbackInputStream
Hi Derek, My changes are finally in place.. I need to do a profiling to see whether they have any good impact on performance... If possible please drop the latest AXIOM binaries (from the nightly build or a manual build from Axiom svn head) in to your test scenerio and see whether if there's an improvment.. Thanks, Thilina On 3/7/07, Thilina Gunarathne <[EMAIL PROTECTED]> wrote: Hi Derek, It's great that you started to nail down the attachment performance issues.. I've also started to do some refactorings to the Attachments handling classes.. I still have some test cases failing., which means it'll take some more time to get the changes in.. Let's crack this together :)... I'll also do some profiling to get more clues.. ~Thilina On 3/7/07, Derek Clayton <[EMAIL PROTECTED]> wrote: > Hi Davanum > > Unfortunately, the server is in .NET so I would only be able to provide the > client and some other sample data which alone might not be useful. > > However, I did some research in to the Axis2 and Axiom source using a client > that was trying to read a ~1.8M binary attachment (XOP/MTOM). It was taking > just over 3 seconds to read in and save the file. I finally tracked down > that 98% of the time involved was occuring in the Class: > > org.apache.axiom.attachments.MIMEBodyPartInputStream specifically in the > read() method at the line: > > // read the next value from stream > int value = inStream.read(); > > The inStream is a PushbackInputStream. On a simple isolated test (i.e. > plain old java reading from the file system) I tested reading in the file > using a BufferedInputStream vs a PushbackInputStream. It took 31 ms for the > BufferedInputStream and a whopping 1047 ms for the PushbackInputStream. (I > know there are reasons for the use of PushbackInputStream) > > I thought I had found the reason why it was slow and even in the Axiom > source just before it creates the PushbackInputStream it states in the Class > org.apache.axiom.attachments.Attachments: > > // do we need to wrap InputStream from a BufferedInputStream before > // wrapping from PushbackStream > > So I changed the source to first wrap in a BufferedInputStream however there > was no change in performance. The immediate Underlying InputStream is a > org.apache.commons.httpclient.AutoCloseInputStream and whatever it might > wrap (etc) I'm not sure. > > So basically I'm still not sure about what's going on. PushbackInputStream > is definitely a lot slower because it does not do any buffered reads. > However, I don't know why the performance wasn't improved when I wrapped it > in a BufferedInputStream. I was unable in my investigative time to find out > any other InputStreams that might have ultimately been wrapped by the > PushbackInputStream. > > Derek > > > >From: "Davanum Srinivas" <[EMAIL PROTECTED]> > >Reply-To: [EMAIL PROTECTED] > >To: axis-user@ws.apache.org > >Subject: Re: MOTM and Encoding/Decoding > >Date: Thu, 1 Mar 2007 14:07:25 -0500 > > > >Derek, > > > >Could you please log a JIRA bug and upload the sample code? Let's try > >to create the scenario on our boxes... > > > >thanks, > >dims > > > >On 3/1/07, Derek Clayton <[EMAIL PROTECTED]> wrote: > >>Thank you for the very fast response. > >> > >> >If possible please post us some numbers of the time comparison. Make > >> >sure to avoid the System.out part when doing the comparison (this > >> >encoding takes time :( )... > >> > >>First let me explain the setup. The two machines are identical > >>hardware...Pentium 4 2.8GHz with 1Gig memory. Software differs since > >>these > >>are two different developer machines. > >> > >>Machine 1 acts as the client and is written in Java using Axis2. It sends > >>an Excel file to Machine 2. When it receives the XML file back (see next > >>paragraph) is simply saves it as a file. > >> > >>Machine 2 acts as the server and is using .NET. It receives an Excel file > >>as binary, saves that file, uses an ActiveX control to read in the file > >>and > >>parse it to generate an XML representation of the data. It then sends > >>that > >>XML back to the Machine 1 in the SOAP response. > >> > >>I haven't written an isolated test but this setup would favor Java/Axis2 > >>anyway since Machine 2 is having to do actual work in addition to reading > >>the binary file. > >> > >>For smaller Excel files the times are quite reasonable. > >> > >>Test 1 > >>-- > >>For a 302K send and a 922K response. > >> > >>Time to send and receive response: 1.6 secs. > >>Time to read response: 6 secs. > >> > >>It is taking about 3 times longer which is reasonable since the file size > >>is > >>3x as large even though the server is doing a lot more work than the > >>client. > >> > >>However, as the files get large the performance begins to break down. > >> > >>Test 2 > >>-- > >>For a 1.4M send and a 5.4M response. > >> > >>Time to send and receive response: 3 secs. > >>Time to read re
Re: MTOM Attachments and PushbackInputStream
Hi Derek, It's great that you started to nail down the attachment performance issues.. I've also started to do some refactorings to the Attachments handling classes.. I still have some test cases failing., which means it'll take some more time to get the changes in.. Let's crack this together :)... I'll also do some profiling to get more clues.. ~Thilina On 3/7/07, Derek Clayton <[EMAIL PROTECTED]> wrote: Hi Davanum Unfortunately, the server is in .NET so I would only be able to provide the client and some other sample data which alone might not be useful. However, I did some research in to the Axis2 and Axiom source using a client that was trying to read a ~1.8M binary attachment (XOP/MTOM). It was taking just over 3 seconds to read in and save the file. I finally tracked down that 98% of the time involved was occuring in the Class: org.apache.axiom.attachments.MIMEBodyPartInputStream specifically in the read() method at the line: // read the next value from stream int value = inStream.read(); The inStream is a PushbackInputStream. On a simple isolated test (i.e. plain old java reading from the file system) I tested reading in the file using a BufferedInputStream vs a PushbackInputStream. It took 31 ms for the BufferedInputStream and a whopping 1047 ms for the PushbackInputStream. (I know there are reasons for the use of PushbackInputStream) I thought I had found the reason why it was slow and even in the Axiom source just before it creates the PushbackInputStream it states in the Class org.apache.axiom.attachments.Attachments: // do we need to wrap InputStream from a BufferedInputStream before // wrapping from PushbackStream So I changed the source to first wrap in a BufferedInputStream however there was no change in performance. The immediate Underlying InputStream is a org.apache.commons.httpclient.AutoCloseInputStream and whatever it might wrap (etc) I'm not sure. So basically I'm still not sure about what's going on. PushbackInputStream is definitely a lot slower because it does not do any buffered reads. However, I don't know why the performance wasn't improved when I wrapped it in a BufferedInputStream. I was unable in my investigative time to find out any other InputStreams that might have ultimately been wrapped by the PushbackInputStream. Derek >From: "Davanum Srinivas" <[EMAIL PROTECTED]> >Reply-To: [EMAIL PROTECTED] >To: axis-user@ws.apache.org >Subject: Re: MOTM and Encoding/Decoding >Date: Thu, 1 Mar 2007 14:07:25 -0500 > >Derek, > >Could you please log a JIRA bug and upload the sample code? Let's try >to create the scenario on our boxes... > >thanks, >dims > >On 3/1/07, Derek Clayton <[EMAIL PROTECTED]> wrote: >>Thank you for the very fast response. >> >> >If possible please post us some numbers of the time comparison. Make >> >sure to avoid the System.out part when doing the comparison (this >> >encoding takes time :( )... >> >>First let me explain the setup. The two machines are identical >>hardware...Pentium 4 2.8GHz with 1Gig memory. Software differs since >>these >>are two different developer machines. >> >>Machine 1 acts as the client and is written in Java using Axis2. It sends >>an Excel file to Machine 2. When it receives the XML file back (see next >>paragraph) is simply saves it as a file. >> >>Machine 2 acts as the server and is using .NET. It receives an Excel file >>as binary, saves that file, uses an ActiveX control to read in the file >>and >>parse it to generate an XML representation of the data. It then sends >>that >>XML back to the Machine 1 in the SOAP response. >> >>I haven't written an isolated test but this setup would favor Java/Axis2 >>anyway since Machine 2 is having to do actual work in addition to reading >>the binary file. >> >>For smaller Excel files the times are quite reasonable. >> >>Test 1 >>-- >>For a 302K send and a 922K response. >> >>Time to send and receive response: 1.6 secs. >>Time to read response: 6 secs. >> >>It is taking about 3 times longer which is reasonable since the file size >>is >>3x as large even though the server is doing a lot more work than the >>client. >> >>However, as the files get large the performance begins to break down. >> >>Test 2 >>-- >>For a 1.4M send and a 5.4M response. >> >>Time to send and receive response: 3 secs. >>Time to read response: 37 secs. >> >>37 seconds to read the 5.4M binary file seems like a long time. As well >>you >>can see that the server processed a larger file (1.4M) in half the time as >>did the client in Test 1. For even larger files the difference becomes >>greater. >> >>Test 3 >>-- >>For a 8.5M and a 32M response. >> >>Time to send and receive response: 18 secs. >>Time to read response: 220 secs. >> >>Here you can see the .NET is reading the 8.5M (along with parsing it and >>generating XML) in 18 secs compared to where Java/Axis2 took 37 secs to >>simply read
Re: MTOM Attachments and PushbackInputStream
Hi Davanum Unfortunately, the server is in .NET so I would only be able to provide the client and some other sample data which alone might not be useful. However, I did some research in to the Axis2 and Axiom source using a client that was trying to read a ~1.8M binary attachment (XOP/MTOM). It was taking just over 3 seconds to read in and save the file. I finally tracked down that 98% of the time involved was occuring in the Class: org.apache.axiom.attachments.MIMEBodyPartInputStream specifically in the read() method at the line: // read the next value from stream int value = inStream.read(); The inStream is a PushbackInputStream. On a simple isolated test (i.e. plain old java reading from the file system) I tested reading in the file using a BufferedInputStream vs a PushbackInputStream. It took 31 ms for the BufferedInputStream and a whopping 1047 ms for the PushbackInputStream. (I know there are reasons for the use of PushbackInputStream) I thought I had found the reason why it was slow and even in the Axiom source just before it creates the PushbackInputStream it states in the Class org.apache.axiom.attachments.Attachments: // do we need to wrap InputStream from a BufferedInputStream before // wrapping from PushbackStream So I changed the source to first wrap in a BufferedInputStream however there was no change in performance. The immediate Underlying InputStream is a org.apache.commons.httpclient.AutoCloseInputStream and whatever it might wrap (etc) I'm not sure. So basically I'm still not sure about what's going on. PushbackInputStream is definitely a lot slower because it does not do any buffered reads. However, I don't know why the performance wasn't improved when I wrapped it in a BufferedInputStream. I was unable in my investigative time to find out any other InputStreams that might have ultimately been wrapped by the PushbackInputStream. Derek From: "Davanum Srinivas" <[EMAIL PROTECTED]> Reply-To: [EMAIL PROTECTED] To: axis-user@ws.apache.org Subject: Re: MOTM and Encoding/Decoding Date: Thu, 1 Mar 2007 14:07:25 -0500 Derek, Could you please log a JIRA bug and upload the sample code? Let's try to create the scenario on our boxes... thanks, dims On 3/1/07, Derek Clayton <[EMAIL PROTECTED]> wrote: Thank you for the very fast response. >If possible please post us some numbers of the time comparison. Make >sure to avoid the System.out part when doing the comparison (this >encoding takes time :( )... First let me explain the setup. The two machines are identical hardware...Pentium 4 2.8GHz with 1Gig memory. Software differs since these are two different developer machines. Machine 1 acts as the client and is written in Java using Axis2. It sends an Excel file to Machine 2. When it receives the XML file back (see next paragraph) is simply saves it as a file. Machine 2 acts as the server and is using .NET. It receives an Excel file as binary, saves that file, uses an ActiveX control to read in the file and parse it to generate an XML representation of the data. It then sends that XML back to the Machine 1 in the SOAP response. I haven't written an isolated test but this setup would favor Java/Axis2 anyway since Machine 2 is having to do actual work in addition to reading the binary file. For smaller Excel files the times are quite reasonable. Test 1 -- For a 302K send and a 922K response. Time to send and receive response: 1.6 secs. Time to read response: 6 secs. It is taking about 3 times longer which is reasonable since the file size is 3x as large even though the server is doing a lot more work than the client. However, as the files get large the performance begins to break down. Test 2 -- For a 1.4M send and a 5.4M response. Time to send and receive response: 3 secs. Time to read response: 37 secs. 37 seconds to read the 5.4M binary file seems like a long time. As well you can see that the server processed a larger file (1.4M) in half the time as did the client in Test 1. For even larger files the difference becomes greater. Test 3 -- For a 8.5M and a 32M response. Time to send and receive response: 18 secs. Time to read response: 220 secs. Here you can see the .NET is reading the 8.5M (along with parsing it and generating XML) in 18 secs compared to where Java/Axis2 took 37 secs to simply read and save a 5.4M file in Test 2. I know this is an informal test case. But is 37 seconds to read a 5.4M binary attachment with no decoding similar to the experience of others? Thanks! Derek _ Don't waste time standing in line—try shopping online. Visit Sympatico / MSN Shopping today! http://shopping.sympatico.msn.ca - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands,
