Thank you for the quick reply. After hearing that the changes should be in
2.2.3 I dug further into my code and found a client-side interceptor which
was causing all attachments to be cached.
I fixed that, and now see a server-side problem when sending attachments.
The server is using SSAJInInterceptor (via WSS4JInInterceptor) and on line
117 of the first there is this:
for (Attachment a : atts) {
if (a.getDataHandler().getDataSource() instanceof AttachmentDataSource)
{
try {
((AttachmentDataSource)a.getDataHandler().getDataSource()).cache();
So the attachment is always being cached. But more problematic is this:
After the above call the “inmem” flag of the CachedOutputStream is correctly
set to false. By the time my application code calls
CachedOutputStram.getInputStream() however, the flag is true, and hence an
empty ByteArrayInputStream is returned.
Is there an apache CXF test case with MTOM enabled, WSS4JInInterceptor used,
and attachment larger than 64k? The one I wrote seems to consistently
fail—the server side application code always getting a non-null, but empty,
stream, when the attachment is larger than 64k.
dkulp wrote:
>
> On Wed September 16 2009 12:31:57 pm K1001010 wrote:
>> Did these changes make it into the 2.2.3 release?
>
> It looks like it, yea.
>
>
>> I am testing with a 150MB attachment, and still see it being streamed to
>> disk
>
> Hmm... I hate to ask this as I don't want a 150MB test case, but is
> there
> any way you could create a "small" test case?
>
> There are a few cases where we do have to buffer it ahead of time. If
> there
> are any SAAJ related things, we need to buffer. Schema validation is
> another.
> If there are multiple attachments, it's likely the others may need to be
> buffered, etc... A testcase would help figure out what is going on.
>
> If you can debug it by setting a breakpoint in
> org.apache.cxf.attachment.AttachementDataSource.cache() and determine why
> cache is being called, that would also be helpful.
>
> Dan
>
>
>
>>
>> I have this code:
>>
>>
>> Picture picture = serviceProxy.getPictuer();
>> DataHandler handler = picture.getImageData();
>> InputStream is = handler.getInputStream();
>>
>>
>> In version 2.2.2 the entire 150BM file would be streamed to temp file on
>> line 3, which would return a file input stream (or a delegate too).
>>
>> In version 2.2.3 the entire 150MB file is streamed to temp file before
>> line
>> 1 returns.
>>
>> I disable schema validation using this system property:
>>
>> org.apache.spring.validation.mode=VALIDATION_NONE
>>
>> Is that correct? Anything else required to stream directly from the
>> wire?
>>
>> Thanks for any help.
>>
>> dkulp wrote:
>> > Well, I THOUGHT this would have been easily doable, but when I checked,
>> > it turns out the attachments were not streaming properly in MTOM mode.
>> > :-(
>> >
>> > I've gone through and fixed a BUNCH of things related to the attachment
>> > handling and streaming and such so some of this is now possible.
>> >
>> > Now, with the latest code, when you get the DataSource, if at all
>> > possible,
>> > the input stream that is returned will properly be streaming off the
>> wire
>> > instead of cached. Thus, you can copy directly from there to the
>> output
>> > stream.
>> >
>> > There are cases where this WON'T work stream directly, notably when
>> > schema validation is turned on. In that case, to work around issues
>> in
>> > JAXB, we buffer the content. Also, if there are multiple
>> attachments,
>> > only the last
>> > one can really be streamed this way. The others get buffered.
>> >
>> > Dan
>> >
>> > On Mon July 27 2009 5:50:00 am dp-isp wrote:
>> >> Hello everybody,
>> >>
>> >> We have a CXF webservice that is a kind of "repository service" and a
>> >> webapp working as frontend for that repository.
>> >> Those two components are on separate hosts.
>> >>
>> >> We would need to realize a continuous stream of data when downloading
>> >> attachments from CXF webservice to the users' browser through the
>> >> frontend
>> >> webapp.
>> >>
>> >> The webapp is basically a servlet that invokes the webservice,
>> receives
>> >> the
>> >> attachment datahandler and copies datahandler InputStream to servlet
>> >> response OutputStream (servlet code is at the end of this email).
>> >>
>> >> Our issue: is it possible to directly link datahandler InputStream to
>> >> servlet response OutputStream without re-building the whole attachment
>> >> in memoroy or in a temporary file? (as can be configured by setting
>> >> org.apache.cxf.io.CachedOutputStream.Threshold,
>> >> org.apache.cxf.io.CachedOutputStream.OutputDirectory)
>> >>
>> >> Going more into details, is there a way to send to CXF the output
>> buffer
>> >> to
>> >> write the attachment on? In such case we would send to CXF the
>> sevlet's
>> >> output buffer.
>> >>
>> >> Alternatively, is there a way to manage CXF working asynchronously
>> with
>> >> attachments, giving back control to the servlet before attachment
>> >> downolad
>> >> completion?
>> >>
>> >> Many thanks,
>> >> Marco
>> >>
>> >> ---
>> >>
>> >> This is the servlet's involved code:
>> >>
>> >> ...
>> >> StreamingService_Service ss = new StreamingService_Service(new
>> >> URL(serviceURL));
>> >> StreamingService port = ss.getServiceImplPort(new MTOMFeature());
>> >>
>> >> Resume myFile = port.downloadMethod(fileName, fileType, usr, pwd,
>> repo);
>> >> DataHandler handler = myFile.getResume();
>> >>
>> >> response.setHeader("Pragma", "No-cache");
>> >> response.setContentType("application/octet-stream");
>> >> response.setHeader("content-disposition", "attachment; filename=" +
>> >> fileName + "." + fileType);
>> >> try {
>> >> InputStream is = handler.getInputStream();
>> >> try {
>> >> OutputStream os = response.getOutputStream();
>> >> try {
>> >> byte[] b = new byte[100000];
>> >> int bytesRead = 0;
>> >> while ((bytesRead = is.read(b)) != -1) {
>> >> os.write(b, 0, bytesRead);
>> >> }
>> >> System.out.println("Download del file " +
>> >> myFile.getCandidateName()
>> +
>> >> "." + myFile.getResumeFileType() + " completato.");
>> >> } finally {
>> >> os.flush();
>> >> os.close();
>> >> }
>> >> } finally {
>> >> is.close();
>> >> }
>> >>
>> >> } catch (Exception e) {
>> >> e.printStackTrace();
>> >> }
>> >> ...
>>
>
> --
> Daniel Kulp
> [email protected]
> http://www.dankulp.com/blog
>
>
--
View this message in context:
http://www.nabble.com/Streaming-large-attachments-by-linking-input-and-output-stream-avoiding-temp-files-tp24677330p25531325.html
Sent from the cxf-user mailing list archive at Nabble.com.
