RE: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
I followed Andreas Veithen's suggestions and created a linked issue in
the WSCOMMONS project:
https://issues.apache.org/jira/browse/WSCOMMONS-318
After downloading the AXIOM 1.2.5 source I found that the fix is
straightforward: change the return type from int to long in the
org.apache.axiom.attachments.Part.getSize() interface (and all other
relevant places) and the transfer of 2.3GB attachments works well.
(I used Axis2 1.3)
Paul Fremantle suggested to cast the DataHandler to a FileDataSource and
then rename the file. This is exactly what we do for files that are
cached to disk by Axis2/AXIOM: if we read the DataHandler as usual the
files would be written to disk twice (once by Axis2/AXIOM and once by
our code). As long as the axis attachmentDir folder and the destination
folder are on the same file system, the File.rename() works well.
Thilina is right: the exception is thrown before we get to the
DataHandler.
Thanks to you all for your help and suggestions!
Crossing my fingers to see the fix in Axis2 1.4...
Regards,
Roy Willy Haug
> -Original Message-
> From: Thilina Gunarathne [mailto:[EMAIL PROTECTED]
> Sent: 1. april 2008 14:00
> To: axis-user@ws.apache.org
> Subject: Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
>
> Hi Paul,
> I don't think this is possible, cause looking at the following
> exception the failure is happening at the parser level.. It never
> returns a DataHandler...
>
> >org.apache.axis2.AxisFault: Referenced Attachment not found in the
> MIME Message.
> >ContentID:1.urn:uuid:[EMAIL PROTECTED]
>
> thanks,
> Thilina
>
> On Tue, Apr 1, 2008 at 3:33 AM, Paul Fremantle <[EMAIL PROTECTED]>
wrote:
> > I believe if the file is cached correctly you can work around this
by
> > not accessing the file via Axiom but simply casting the DataSource
as
> > a FileDataSource and then working on the file directly. Of course
we
> > still need to fix Axiom!
> >
> > DataHandler dataHandler = (DataHandler)omText.getDataHandler();
> > FileDataSource fileDataSource =
> (FileDataSource)dataHandler.getDataSource();
> > File file = fileDataSource.getFile();
> > file.renameTo(new File("MyFile.bin"));
> >
> >
> > Paul
> >
> >
> >
> > On Tue, Apr 1, 2008 at 9:08 AM, Roy Willy Haug
> > <[EMAIL PROTECTED]> wrote:
> > > I have created an issue for this in Jira:
> > >
> > > https://issues.apache.org/jira/browse/AXIS2-3686
> > >
> > > (I set fix version to 1.4, although I realize it might be too
late.
> :-)
> > > )
> > >
> > > Regards Roy Willy Haug
> > >
> > >
> > >
> > >
> > >
> > > > -Original Message-
> > > > From: Thilina Gunarathne [mailto:[EMAIL PROTECTED]
> > > > Sent: 1. april 2008 00:29
> > > > To: axis-user@ws.apache.org
> > > > Subject: Re: MTOM attachments larger than 2.1GB
> (Integer.MAX_VALUE?)
> > > >
> > > > Please log a Jira with all these details...
> > > >
> > > > I think the fix needs start from the FileAccessor getSize()
> onwards..
> > > >
> > > > thanks,
> > > > Thilina
> > > >
> > > > On Mon, Mar 31, 2008 at 12:01 PM, Andreas Veithen
> > > > <[EMAIL PROTECTED]> wrote:
> > > > > Roy,
> > > > >
> > > > > This seems to be a limitation of AXIOM that uses the int
type
> for
> > > all
> > > > > length variables in the
> org.apache.axiom.attachments.Attachments
> > > class
> > > > > and various other classes it depends on.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Andreas
> > > > >
> > > > >
> > > > >
> > > > > Quoting Roy Willy Haug <[EMAIL PROTECTED]>:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > >
> > > > > >
> > > > > > We have a well working file transfer service that works
with
> > > files up
> > > > to
> > > > > > approximately 2.1 GB (probably Integer.MAX_VALUE:
2147483647
> > > bytes?).
> > > > > >
> > > > > >
> > > > > >
> > > > > > When transferring even larger files, Axis2 fails with the
> > > following
> > > > &
Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
Hi Paul,
I don't think this is possible, cause looking at the following
exception the failure is happening at the parser level.. It never
returns a DataHandler...
>org.apache.axis2.AxisFault: Referenced Attachment not found in the
MIME Message. >ContentID:1.urn:uuid:[EMAIL PROTECTED]
thanks,
Thilina
On Tue, Apr 1, 2008 at 3:33 AM, Paul Fremantle <[EMAIL PROTECTED]> wrote:
> I believe if the file is cached correctly you can work around this by
> not accessing the file via Axiom but simply casting the DataSource as
> a FileDataSource and then working on the file directly. Of course we
> still need to fix Axiom!
>
> DataHandler dataHandler = (DataHandler)omText.getDataHandler();
> FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
> File file = fileDataSource.getFile();
> file.renameTo(new File("MyFile.bin"));
>
>
> Paul
>
>
>
> On Tue, Apr 1, 2008 at 9:08 AM, Roy Willy Haug
> <[EMAIL PROTECTED]> wrote:
> > I have created an issue for this in Jira:
> >
> > https://issues.apache.org/jira/browse/AXIS2-3686
> >
> > (I set fix version to 1.4, although I realize it might be too late. :-)
> > )
> >
> > Regards Roy Willy Haug
> >
> >
> >
> >
> >
> > > -----Original Message-----
> > > From: Thilina Gunarathne [mailto:[EMAIL PROTECTED]
> > > Sent: 1. april 2008 00:29
> > > To: axis-user@ws.apache.org
> > > Subject: Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
> > >
> > > Please log a Jira with all these details...
> > >
> > > I think the fix needs start from the FileAccessor getSize() onwards..
> > >
> > > thanks,
> > > Thilina
> > >
> > > On Mon, Mar 31, 2008 at 12:01 PM, Andreas Veithen
> > > <[EMAIL PROTECTED]> wrote:
> > > > Roy,
> > > >
> > > > This seems to be a limitation of AXIOM that uses the int type for
> > all
> > > > length variables in the org.apache.axiom.attachments.Attachments
> > class
> > > > and various other classes it depends on.
> > > >
> > > > Regards,
> > > >
> > > > Andreas
> > > >
> > > >
> > > >
> > > > Quoting Roy Willy Haug <[EMAIL PROTECTED]>:
> > > >
> > > > > Hi,
> > > > >
> > > > >
> > > > >
> > > > > We have a well working file transfer service that works with
> > files up
> > > to
> > > > > approximately 2.1 GB (probably Integer.MAX_VALUE: 2147483647
> > bytes?).
> > > > >
> > > > >
> > > > >
> > > > > When transferring even larger files, Axis2 fails with the
> > following
> > > > > message:
> > > > >
> > > > >
> > > > >
> > > > > org.apache.axis2.AxisFault: Referenced Attachment not found
> > in
> > > the
> > > > > MIME Message.
> > > > > ContentID:1.urn:uuid:[EMAIL PROTECTED]
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > What happens is:
> > > > >
> > > > >
> > > > >
> > > > > The client transfers the complete file to the server.
> > > > >
> > > > > The server correctly caches the file in the attachmentDIR folder.
> > > > >
> > > > > The AxisFault is thrown.
> > > > >
> > > > >
> > > > >
> > > > > The server-side cached file have the exact same byte-length as
> > the
> > > file
> > > > > the client sent, so every byte has been transferred.
> > > > >
> > > > >
> > > > >
> > > > > I am currently using Windows XP with NTFS and support for files
> > much
> > > > > larger than 2.1 GB on both sides, Axis2 1.3, document-literal
> > style,
> > > > > ADB, MTOM and caching to disk.
> > > > >
> > > > >
> > > > >
> > > > > Any ideas?
> > > > >
> > > > >
> > > > >
> > > > > The Data
RE: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
Since the problem comes from Axiom (Axis2 passes the length as String to Axiom), actually the JIRA should be in the WSCOMMONS project. I don't know if it is possible to move an existing JIRA issue to another project. Otherwise we should open a new JIRA in WSCOMMONS and link it to the AXIS2 issue you created. Andreas Quoting Roy Willy Haug <[EMAIL PROTECTED]>: I have created an issue for this in Jira: https://issues.apache.org/jira/browse/AXIS2-3686 (I set fix version to 1.4, although I realize it might be too late. :-) ) Regards Roy Willy Haug -Original Message- From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] Sent: 1. april 2008 00:29 To: axis-user@ws.apache.org Subject: Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?) Please log a Jira with all these details... I think the fix needs start from the FileAccessor getSize() onwards.. thanks, Thilina On Mon, Mar 31, 2008 at 12:01 PM, Andreas Veithen <[EMAIL PROTECTED]> wrote: > Roy, > > This seems to be a limitation of AXIOM that uses the int type for all > length variables in the org.apache.axiom.attachments.Attachments class > and various other classes it depends on. > > Regards, > > Andreas > > > > Quoting Roy Willy Haug <[EMAIL PROTECTED]>: > > > Hi, > > > > > > > > We have a well working file transfer service that works with files up to > > approximately 2.1 GB (probably Integer.MAX_VALUE: 2147483647 bytes?). > > > > > > > > When transferring even larger files, Axis2 fails with the following > > message: > > > > > > > > org.apache.axis2.AxisFault: Referenced Attachment not found in the > > MIME Message. > > ContentID:1.urn:uuid:[EMAIL PROTECTED] > > > > > > > > > > > > What happens is: > > > > > > > > The client transfers the complete file to the server. > > > > The server correctly caches the file in the attachmentDIR folder. > > > > The AxisFault is thrown. > > > > > > > > The server-side cached file have the exact same byte-length as the file > > the client sent, so every byte has been transferred. > > > > > > > > I am currently using Windows XP with NTFS and support for files much > > larger than 2.1 GB on both sides, Axis2 1.3, document-literal style, > > ADB, MTOM and caching to disk. > > > > > > > > Any ideas? > > > > > > > > The DataHandler that is supplied to the client stub is generated like > > this: > > > > FileDataSource datasource = new FileDataSource(localFile); > > > > dh = new DataHandler(datasource); > > > > > > > > > > > > Parts of my Axis2.xml file: > > > > true > > > > false > > > > true > > > > C:/TEMP/da_server_temp/ > > > > 5 > > > > > > > > > > > > Regards, > > > > > > > > Roy Willy Haug > > > > > > > > > > > > > > > > > > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Thilina Gunarathne - http://thilinag.blogspot.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
I believe if the file is cached correctly you can work around this by
not accessing the file via Axiom but simply casting the DataSource as
a FileDataSource and then working on the file directly. Of course we
still need to fix Axiom!
DataHandler dataHandler = (DataHandler)omText.getDataHandler();
FileDataSource fileDataSource = (FileDataSource)dataHandler.getDataSource();
File file = fileDataSource.getFile();
file.renameTo(new File("MyFile.bin"));
Paul
On Tue, Apr 1, 2008 at 9:08 AM, Roy Willy Haug
<[EMAIL PROTECTED]> wrote:
> I have created an issue for this in Jira:
>
> https://issues.apache.org/jira/browse/AXIS2-3686
>
> (I set fix version to 1.4, although I realize it might be too late. :-)
> )
>
> Regards Roy Willy Haug
>
>
>
>
>
> > -Original Message-
> > From: Thilina Gunarathne [mailto:[EMAIL PROTECTED]
> > Sent: 1. april 2008 00:29
> > To: axis-user@ws.apache.org
> > Subject: Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
> >
> > Please log a Jira with all these details...
> >
> > I think the fix needs start from the FileAccessor getSize() onwards..
> >
> > thanks,
> > Thilina
> >
> > On Mon, Mar 31, 2008 at 12:01 PM, Andreas Veithen
> > <[EMAIL PROTECTED]> wrote:
> > > Roy,
> > >
> > > This seems to be a limitation of AXIOM that uses the int type for
> all
> > > length variables in the org.apache.axiom.attachments.Attachments
> class
> > > and various other classes it depends on.
> > >
> > > Regards,
> > >
> > > Andreas
> > >
> > >
> > >
> > > Quoting Roy Willy Haug <[EMAIL PROTECTED]>:
> > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > We have a well working file transfer service that works with
> files up
> > to
> > > > approximately 2.1 GB (probably Integer.MAX_VALUE: 2147483647
> bytes?).
> > > >
> > > >
> > > >
> > > > When transferring even larger files, Axis2 fails with the
> following
> > > > message:
> > > >
> > > >
> > > >
> > > > org.apache.axis2.AxisFault: Referenced Attachment not found
> in
> > the
> > > > MIME Message.
> > > > ContentID:1.urn:uuid:[EMAIL PROTECTED]
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > What happens is:
> > > >
> > > >
> > > >
> > > > The client transfers the complete file to the server.
> > > >
> > > > The server correctly caches the file in the attachmentDIR folder.
> > > >
> > > > The AxisFault is thrown.
> > > >
> > > >
> > > >
> > > > The server-side cached file have the exact same byte-length as
> the
> > file
> > > > the client sent, so every byte has been transferred.
> > > >
> > > >
> > > >
> > > > I am currently using Windows XP with NTFS and support for files
> much
> > > > larger than 2.1 GB on both sides, Axis2 1.3, document-literal
> style,
> > > > ADB, MTOM and caching to disk.
> > > >
> > > >
> > > >
> > > > Any ideas?
> > > >
> > > >
> > > >
> > > > The DataHandler that is supplied to the client stub is generated
> like
> > > > this:
> > > >
> > > > FileDataSource datasource = new
> FileDataSource(localFile);
> > > >
> > > > dh = new DataHandler(datasource);
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Parts of my Axis2.xml file:
> > > >
> > > > true
> > > >
> > > > false
> > > >
> > > > true
> > > >
> > > > > name="attachmentDIR">C:/TEMP/da_server_temp/
> > > >
> > > > 5
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > >
> > > >
> > > > Roy Willy Haug
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > >
> > >
> -
> > > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > > For additional commands, e-mail: [EMAIL PROTECTED]
> > >
> > >
> >
> >
> >
> > --
> > Thilina Gunarathne - http://thilinag.blogspot.com
> >
> > -
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
--
Paul Fremantle
Co-Founder and VP of Technical Sales, WSO2
Apache Synapse PMC Chair
OASIS WS-RX TC Co-chair
blog: http://pzf.fremantle.org
[EMAIL PROTECTED]
"Oxygenating the Web Service Platform", www.wso2.com
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
I have created an issue for this in Jira: https://issues.apache.org/jira/browse/AXIS2-3686 (I set fix version to 1.4, although I realize it might be too late. :-) ) Regards Roy Willy Haug > -Original Message- > From: Thilina Gunarathne [mailto:[EMAIL PROTECTED] > Sent: 1. april 2008 00:29 > To: axis-user@ws.apache.org > Subject: Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?) > > Please log a Jira with all these details... > > I think the fix needs start from the FileAccessor getSize() onwards.. > > thanks, > Thilina > > On Mon, Mar 31, 2008 at 12:01 PM, Andreas Veithen > <[EMAIL PROTECTED]> wrote: > > Roy, > > > > This seems to be a limitation of AXIOM that uses the int type for all > > length variables in the org.apache.axiom.attachments.Attachments class > > and various other classes it depends on. > > > > Regards, > > > > Andreas > > > > > > > > Quoting Roy Willy Haug <[EMAIL PROTECTED]>: > > > > > Hi, > > > > > > > > > > > > We have a well working file transfer service that works with files up > to > > > approximately 2.1 GB (probably Integer.MAX_VALUE: 2147483647 bytes?). > > > > > > > > > > > > When transferring even larger files, Axis2 fails with the following > > > message: > > > > > > > > > > > > org.apache.axis2.AxisFault: Referenced Attachment not found in > the > > > MIME Message. > > > ContentID:1.urn:uuid:[EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > What happens is: > > > > > > > > > > > > The client transfers the complete file to the server. > > > > > > The server correctly caches the file in the attachmentDIR folder. > > > > > > The AxisFault is thrown. > > > > > > > > > > > > The server-side cached file have the exact same byte-length as the > file > > > the client sent, so every byte has been transferred. > > > > > > > > > > > > I am currently using Windows XP with NTFS and support for files much > > > larger than 2.1 GB on both sides, Axis2 1.3, document-literal style, > > > ADB, MTOM and caching to disk. > > > > > > > > > > > > Any ideas? > > > > > > > > > > > > The DataHandler that is supplied to the client stub is generated like > > > this: > > > > > > FileDataSource datasource = new FileDataSource(localFile); > > > > > > dh = new DataHandler(datasource); > > > > > > > > > > > > > > > > > > Parts of my Axis2.xml file: > > > > > > true > > > > > > false > > > > > > true > > > > > > name="attachmentDIR">C:/TEMP/da_server_temp/ > > > > > > 5 > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > Roy Willy Haug > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > -- > Thilina Gunarathne - http://thilinag.blogspot.com > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
Please log a Jira with all these details... I think the fix needs start from the FileAccessor getSize() onwards.. thanks, Thilina On Mon, Mar 31, 2008 at 12:01 PM, Andreas Veithen <[EMAIL PROTECTED]> wrote: > Roy, > > This seems to be a limitation of AXIOM that uses the int type for all > length variables in the org.apache.axiom.attachments.Attachments class > and various other classes it depends on. > > Regards, > > Andreas > > > > Quoting Roy Willy Haug <[EMAIL PROTECTED]>: > > > Hi, > > > > > > > > We have a well working file transfer service that works with files up to > > approximately 2.1 GB (probably Integer.MAX_VALUE: 2147483647 bytes?). > > > > > > > > When transferring even larger files, Axis2 fails with the following > > message: > > > > > > > > org.apache.axis2.AxisFault: Referenced Attachment not found in the > > MIME Message. > > ContentID:1.urn:uuid:[EMAIL PROTECTED] > > > > > > > > > > > > What happens is: > > > > > > > > The client transfers the complete file to the server. > > > > The server correctly caches the file in the attachmentDIR folder. > > > > The AxisFault is thrown. > > > > > > > > The server-side cached file have the exact same byte-length as the file > > the client sent, so every byte has been transferred. > > > > > > > > I am currently using Windows XP with NTFS and support for files much > > larger than 2.1 GB on both sides, Axis2 1.3, document-literal style, > > ADB, MTOM and caching to disk. > > > > > > > > Any ideas? > > > > > > > > The DataHandler that is supplied to the client stub is generated like > > this: > > > > FileDataSource datasource = new FileDataSource(localFile); > > > > dh = new DataHandler(datasource); > > > > > > > > > > > > Parts of my Axis2.xml file: > > > > true > > > > false > > > > true > > > > C:/TEMP/da_server_temp/ > > > > 5 > > > > > > > > > > > > Regards, > > > > > > > > Roy Willy Haug > > > > > > > > > > > > > > > > > > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Thilina Gunarathne - http://thilinag.blogspot.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: MTOM attachments larger than 2.1GB (Integer.MAX_VALUE?)
Roy, This seems to be a limitation of AXIOM that uses the int type for all length variables in the org.apache.axiom.attachments.Attachments class and various other classes it depends on. Regards, Andreas Quoting Roy Willy Haug <[EMAIL PROTECTED]>: Hi, We have a well working file transfer service that works with files up to approximately 2.1 GB (probably Integer.MAX_VALUE: 2147483647 bytes?). When transferring even larger files, Axis2 fails with the following message: org.apache.axis2.AxisFault: Referenced Attachment not found in the MIME Message. ContentID:1.urn:uuid:[EMAIL PROTECTED] What happens is: The client transfers the complete file to the server. The server correctly caches the file in the attachmentDIR folder. The AxisFault is thrown. The server-side cached file have the exact same byte-length as the file the client sent, so every byte has been transferred. I am currently using Windows XP with NTFS and support for files much larger than 2.1 GB on both sides, Axis2 1.3, document-literal style, ADB, MTOM and caching to disk. Any ideas? The DataHandler that is supplied to the client stub is generated like this: FileDataSource datasource = new FileDataSource(localFile); dh = new DataHandler(datasource); Parts of my Axis2.xml file: true false true C:/TEMP/da_server_temp/ 5 Regards, Roy Willy Haug - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
