Just to let you know, I was finally able to reproduce this, but it wasn't
easy. I added the SAAJ interceptor to our mtom test case and increased the
data size to 250K, but I still couldn't reproduce the error in eclipse.
However, when I ran mvn on the command line it failed, but not with Java 6,
only Java 5. I really have no idea why I cannot reliably reproduce it,
especially in eclipse. Doesn't matter what JDK I use in eclipse, I cannot
reproduce it there. Makes debugging hard.
In anycase, by adding a bunch of Thread.dumpStacks and printlns and such
throughout the code, I THINK I've tracked down what is causing this. I'll
run some more tests and and get changes committed up today, I hope. I
mostly need to go through and remove all my dumpStacks and printlns now as
well. :-(
Dan
On Wed September 23 2009 7:11:56 pm K1001010 wrote:
> Thank you for the quick reply. After hearing that the changes should be in
> 2.2.3 I dug further into my code and found a client-side interceptor which
> was causing all attachments to be cached.
>
> I fixed that, and now see a server-side problem when sending attachments.
> The server is using SSAJInInterceptor (via WSS4JInInterceptor) and on line
> 117 of the first there is this:
>
> for (Attachment a : atts) {
> if (a.getDataHandler().getDataSource() instanceof
> AttachmentDataSource) {
> try {
>
> ((AttachmentDataSource)a.getDataHandler().getDataSource()).cache();
>
>
> So the attachment is always being cached. But more problematic is this:
>
> After the above call the “inmem” flag of the CachedOutputStream is
> correctly set to false. By the time my application code calls
> CachedOutputStram.getInputStream() however, the flag is true, and hence an
> empty ByteArrayInputStream is returned.
>
> Is there an apache CXF test case with MTOM enabled, WSS4JInInterceptor
> used, and attachment larger than 64k? The one I wrote seems to
> consistently fail—the server side application code always getting a
> non-null, but empty, stream, when the attachment is larger than 64k.
>
> dkulp wrote:
> > On Wed September 16 2009 12:31:57 pm K1001010 wrote:
> >> Did these changes make it into the 2.2.3 release?
> >
> > It looks like it, yea.
> >
> >> I am testing with a 150MB attachment, and still see it being streamed to
> >> disk
> >
> > Hmm... I hate to ask this as I don't want a 150MB test case, but is
> > there
> > any way you could create a "small" test case?
> >
> > There are a few cases where we do have to buffer it ahead of time. If
> > there
> > are any SAAJ related things, we need to buffer. Schema validation is
> > another.
> > If there are multiple attachments, it's likely the others may need to be
> > buffered, etc... A testcase would help figure out what is going on.
> >
> > If you can debug it by setting a breakpoint in
> > org.apache.cxf.attachment.AttachementDataSource.cache() and determine why
> > cache is being called, that would also be helpful.
> >
> > Dan
> >
> >> I have this code:
> >>
> >>
> >> Picture picture = serviceProxy.getPictuer();
> >> DataHandler handler = picture.getImageData();
> >> InputStream is = handler.getInputStream();
> >>
> >>
> >> In version 2.2.2 the entire 150BM file would be streamed to temp file on
> >> line 3, which would return a file input stream (or a delegate too).
> >>
> >> In version 2.2.3 the entire 150MB file is streamed to temp file before
> >> line
> >> 1 returns.
> >>
> >> I disable schema validation using this system property:
> >>
> >> org.apache.spring.validation.mode=VALIDATION_NONE
> >>
> >> Is that correct? Anything else required to stream directly from the
> >> wire?
> >>
> >> Thanks for any help.
> >>
> >> dkulp wrote:
> >> > Well, I THOUGHT this would have been easily doable, but when I
> >> > checked, it turns out the attachments were not streaming properly in
> >> > MTOM mode.
> >> >
> >> > :-(
> >> >
> >> > I've gone through and fixed a BUNCH of things related to the
> >> > attachment handling and streaming and such so some of this is now
> >> > possible.
> >> >
> >> > Now, with the latest code, when you get the DataSource, if at all
> >> > possible,
> >> > the input stream that is returned will properly be streaming off the
> >>
> >> wire
> >>
> >> > instead of cached. Thus, you can copy directly from there to the
> >>
> >> output
> >>
> >> > stream.
> >> >
> >> > There are cases where this WON'T work stream directly, notably when
> >> > schema validation is turned on. In that case, to work around issues
> >>
> >> in
> >>
> >> > JAXB, we buffer the content. Also, if there are multiple
> >>
> >> attachments,
> >>
> >> > only the last
> >> > one can really be streamed this way. The others get buffered.
> >> >
> >> > Dan
> >> >
> >> > On Mon July 27 2009 5:50:00 am dp-isp wrote:
> >> >> Hello everybody,
> >> >>
> >> >> We have a CXF webservice that is a kind of "repository service" and a
> >> >> webapp working as frontend for that repository.
> >> >> Those two components are on separate hosts.
> >> >>
> >> >> We would need to realize a continuous stream of data when downloading
> >> >> attachments from CXF webservice to the users' browser through the
> >> >> frontend
> >> >> webapp.
> >> >>
> >> >> The webapp is basically a servlet that invokes the webservice,
> >>
> >> receives
> >>
> >> >> the
> >> >> attachment datahandler and copies datahandler InputStream to servlet
> >> >> response OutputStream (servlet code is at the end of this email).
> >> >>
> >> >> Our issue: is it possible to directly link datahandler InputStream to
> >> >> servlet response OutputStream without re-building the whole
> >> >> attachment in memoroy or in a temporary file? (as can be configured
> >> >> by setting org.apache.cxf.io.CachedOutputStream.Threshold,
> >> >> org.apache.cxf.io.CachedOutputStream.OutputDirectory)
> >> >>
> >> >> Going more into details, is there a way to send to CXF the output
> >>
> >> buffer
> >>
> >> >> to
> >> >> write the attachment on? In such case we would send to CXF the
> >>
> >> sevlet's
> >>
> >> >> output buffer.
> >> >>
> >> >> Alternatively, is there a way to manage CXF working asynchronously
> >>
> >> with
> >>
> >> >> attachments, giving back control to the servlet before attachment
> >> >> downolad
> >> >> completion?
> >> >>
> >> >> Many thanks,
> >> >> Marco
> >> >>
> >> >> ---
> >> >>
> >> >> This is the servlet's involved code:
> >> >>
> >> >> ...
> >> >> StreamingService_Service ss = new StreamingService_Service(new
> >> >> URL(serviceURL));
> >> >> StreamingService port = ss.getServiceImplPort(new MTOMFeature());
> >> >>
> >> >> Resume myFile = port.downloadMethod(fileName, fileType, usr, pwd,
> >>
> >> repo);
> >>
> >> >> DataHandler handler = myFile.getResume();
> >> >>
> >> >> response.setHeader("Pragma", "No-cache");
> >> >> response.setContentType("application/octet-stream");
> >> >> response.setHeader("content-disposition", "attachment; filename=" +
> >> >> fileName + "." + fileType);
> >> >> try {
> >> >> InputStream is = handler.getInputStream();
> >> >> try {
> >> >> OutputStream os = response.getOutputStream();
> >> >> try {
> >> >> byte[] b = new byte[100000];
> >> >> int bytesRead = 0;
> >> >> while ((bytesRead = is.read(b)) != -1) {
> >> >> os.write(b, 0, bytesRead);
> >> >> }
> >> >> System.out.println("Download del file " +
> >> >> myFile.getCandidateName()
> >>
> >> +
> >>
> >> >> "." + myFile.getResumeFileType() + " completato.");
> >> >> } finally {
> >> >> os.flush();
> >> >> os.close();
> >> >> }
> >> >> } finally {
> >> >> is.close();
> >> >> }
> >> >>
> >> >> } catch (Exception e) {
> >> >> e.printStackTrace();
> >> >> }
> >> >> ...
>
--
Daniel Kulp
[email protected]
http://www.dankulp.com/blog
