Thank you for the quick reply. After hearing that the changes should be
in 2.2.3 I dug further into my code and found a client-side interceptor
which was causing all attachments to be cached.
Is there an apache CXF test case with MTOM enabled, WSS4JInInterceptor
used, and attachment larger than 64k? The one I wrote seems to
consistently fail-the server side application code always getting a
non-null, but empty, stream when the attachment is larger than 64k.
The server is using SSAJInInterceptor (via WSS4JInInterceptor) and on
line 117 of the first there is this:
for (Attachment a : atts) {
if (a.getDataHandler().getDataSource() instanceof
AttachmentDataSource) {
try {
((AttachmentDataSource)a.getDataHandler().getDataSource()).cache();
So the attachment is always being cached. But more problematic is this:
After the above call the "inmem" flag of the CachedOutputStream is
correctly set to false. By the time my application code calls
CachedOutputStram.getInputStream() however, the flag is true, and hence
an empty ByteArrayInputStream is returned.
Thanks for any help.
-----Original Message-----
From: Daniel Kulp [mailto:[email protected]]
Sent: Wednesday, September 16, 2009 9:33 PM
To: [email protected]
Cc: WINKLER, KEITH DAVID [AG-Contractor/1000]
Subject: Re: Streaming large attachments by linking input and output
stream avoiding temp files
On Wed September 16 2009 12:31:57 pm K1001010 wrote:
> Did these changes make it into the 2.2.3 release?
It looks like it, yea.
> I am testing with a 150MB attachment, and still see it being streamed
to
> disk
Hmm... I hate to ask this as I don't want a 150MB test case, but is
there
any way you could create a "small" test case?
There are a few cases where we do have to buffer it ahead of time. If
there
are any SAAJ related things, we need to buffer. Schema validation is
another.
If there are multiple attachments, it's likely the others may need to be
buffered, etc... A testcase would help figure out what is going on.
If you can debug it by setting a breakpoint in
org.apache.cxf.attachment.AttachementDataSource.cache() and determine
why
cache is being called, that would also be helpful.
Dan
>
> I have this code:
>
>
> Picture picture = serviceProxy.getPictuer();
> DataHandler handler = picture.getImageData();
> InputStream is = handler.getInputStream();
>
>
> In version 2.2.2 the entire 150BM file would be streamed to temp file
on
> line 3, which would return a file input stream (or a delegate too).
>
> In version 2.2.3 the entire 150MB file is streamed to temp file before
line
> 1 returns.
>
> I disable schema validation using this system property:
>
> org.apache.spring.validation.mode=VALIDATION_NONE
>
> Is that correct? Anything else required to stream directly from the
wire?
>
> Thanks for any help.
>
> dkulp wrote:
> > Well, I THOUGHT this would have been easily doable, but when I
checked,
> > it turns out the attachments were not streaming properly in MTOM
mode.
> > :-(
> >
> > I've gone through and fixed a BUNCH of things related to the
attachment
> > handling and streaming and such so some of this is now possible.
> >
> > Now, with the latest code, when you get the DataSource, if at all
> > possible,
> > the input stream that is returned will properly be streaming off the
wire
> > instead of cached. Thus, you can copy directly from there to the
output
> > stream.
> >
> > There are cases where this WON'T work stream directly, notably when
> > schema validation is turned on. In that case, to work around
issues in
> > JAXB, we buffer the content. Also, if there are multiple
attachments,
> > only the last
> > one can really be streamed this way. The others get buffered.
> >
> > Dan
> >
> > On Mon July 27 2009 5:50:00 am dp-isp wrote:
> >> Hello everybody,
> >>
> >> We have a CXF webservice that is a kind of "repository service" and
a
> >> webapp working as frontend for that repository.
> >> Those two components are on separate hosts.
> >>
> >> We would need to realize a continuous stream of data when
downloading
> >> attachments from CXF webservice to the users' browser through the
> >> frontend
> >> webapp.
> >>
> >> The webapp is basically a servlet that invokes the webservice,
receives
> >> the
> >> attachment datahandler and copies datahandler InputStream to
servlet
> >> response OutputStream (servlet code is at the end of this email).
> >>
> >> Our issue: is it possible to directly link datahandler InputStream
to
> >> servlet response OutputStream without re-building the whole
attachment
> >> in memoroy or in a temporary file? (as can be configured by setting
> >> org.apache.cxf.io.CachedOutputStream.Threshold,
> >> org.apache.cxf.io.CachedOutputStream.OutputDirectory)
> >>
> >> Going more into details, is there a way to send to CXF the output
buffer
> >> to
> >> write the attachment on? In such case we would send to CXF the
sevlet's
> >> output buffer.
> >>
> >> Alternatively, is there a way to manage CXF working asynchronously
with
> >> attachments, giving back control to the servlet before attachment
> >> downolad
> >> completion?
> >>
> >> Many thanks,
> >> Marco
> >>
> >> ---
> >>
> >> This is the servlet's involved code:
> >>
> >> ...
> >> StreamingService_Service ss = new StreamingService_Service(new
> >> URL(serviceURL));
> >> StreamingService port = ss.getServiceImplPort(new MTOMFeature());
> >>
> >> Resume myFile = port.downloadMethod(fileName, fileType, usr, pwd,
repo);
> >> DataHandler handler = myFile.getResume();
> >>
> >> response.setHeader("Pragma", "No-cache");
> >> response.setContentType("application/octet-stream");
> >> response.setHeader("content-disposition", "attachment; filename=" +
> >> fileName + "." + fileType);
> >> try {
> >> InputStream is = handler.getInputStream();
> >> try {
> >> OutputStream os = response.getOutputStream();
> >> try {
> >> byte[] b = new byte[100000];
> >> int bytesRead = 0;
> >> while ((bytesRead = is.read(b)) != -1) {
> >> os.write(b, 0, bytesRead);
> >> }
> >> System.out.println("Download del file " +
myFile.getCandidateName() +
> >> "." + myFile.getResumeFileType() + " completato.");
> >> } finally {
> >> os.flush();
> >> os.close();
> >> }
> >> } finally {
> >> is.close();
> >> }
> >>
> >> } catch (Exception e) {
> >> e.printStackTrace();
> >> }
> >> ...
>
--
Daniel Kulp
[email protected]
http://www.dankulp.com/blog
---------------------------------------------------------------------------------------------------------
This e-mail message may contain privileged and/or confidential information, and
is intended to be received only by persons entitled to receive such
information. If you have received this e-mail in error, please notify the
sender immediately. Please delete it and all attachments from any servers, hard
drives or any other media. Other use of this e-mail by you is strictly
prohibited.
All e-mails and attachments sent and received are subject to monitoring,
reading and archival by Monsanto, including its subsidiaries. The recipient of
this e-mail is solely responsible for checking for the presence of "Viruses" or
other "Malware". Monsanto, along with its subsidiaries, accepts no liability
for any damage caused by any such code transmitted by or accompanying this
e-mail or any attachment.
---------------------------------------------------------------------------------------------------------
