> > But the > problem with this approach is when the SOAP part is pretty large the > whole message (I mean SOAP) is in memory. Is the same thing happens with > Axis2/JAVA ?
IIRC Axis2/java does the combination I mentioned above. Use stream as much as possible, then buffer it.. Also in Axis2/java even the SOAP part can get file cached if it's very large, in which case it will not be in memory. In any case I would not expect a huge SOAP envelope, specially when attachments are used.. thanks, Thilina > > Thanks, > -Manjula. > > On Tue, 2008-07-15 at 03:26 -0700, Thilina Gunarathne wrote: > > Hi Manjula, > > One thing you guys can try to do is to break up the stream at the > > MIME() parser level. Then the MIME parser only gives the SOAP Envelope > > part to the XML parser and nothing beyond. This can be achieved by > > faking a input stream. MIME parser gets the original input stream and > > then gives the XML parser a faked inputstream, something which > > provides the same interface and functionality as the original > > inputstream, but the MIME parser has control over what goes in to > > it. > > > > For an example, if reading an InputStream is reading a memory pointer > > (please excuse my very bad knowledge about C I/O) which gets filled by > > the transport layer, then the MIME parser can also write the data in > > to some memory location which will act as the input stream location > > for XMLReader. > > > > If the above is possible, then the MIME parser can stream the actual > > input stream directly to the faked inputstream till the SOAP part > > boundary is reached or till the user requests another MIME part. > > Streaming MIME parsing will happen throughout the process to limit the > > FAKED input stream to the contents of the SOAP part. If the user > > requests a MIME part while the XML reader is only half way in to > > parsing the SOAP envelope, then the MIME parser should read ahead the > > SOAP part, store it in memory and should provide it as needed to the > > FAKED input stream. One other way to visualize this is to think MIME > > parsing happens at a level below the XML parsing, just like transport > > level. > > > > I'm not sure how hard to do this in Axis2/C (or whether it's > > impossible)... Let me know if I'm not clear with the idea. > > > > thanks, > > Thilina > > > > > > On Tue, Jul 15, 2008 at 3:04 AM, Supun Kamburugamuva > > <[EMAIL PROTECTED]> wrote: > > > > But then again the problem will occur when xml_parser > > give the control > > back to mime_parser. Because xml_parser will read > > beyond the soap part > > when parsing. So the parser should be modified to > > return the the > > unwanted data it read beyond the soap part while > > parsing. I think > > Guththila can be modified for this requirement, but > > does any one know > > how to do this with libxml2. > > > > I assume by "read beyond the soap part" you mean consuming the > > stream beyond the soap part. But I think this is unavoidable > > in any parser. > > > > 1. Parser don't know where the xml ends. > > 2. The parser doesn't read character by chareacter from the > > stream (if it does it'll be very inefficient) and it reads > > chunks of data from the stream. > > > > Becuase of the above reasons the parser will consume the > > stream beyond the soap part. > > > > Supun.. > > > > > > > > > > > > On Sat, 2008-07-12 at 21:21 +0530, Manjula Peiris > > wrote: > > > On Sat, 2008-07-12 at 19:31 +0530, Milinda Pathirage > > wrote: > > > > As Thilina mentioned in above thread Axis2/Java > > keeps a instance of > > > > mime parser in it's xml builder. So I think it's > > ok to follow that > > > > method. AFAIK, I think if we enable MTOM caching > > we have to have this > > > > feature or is anybody have ideas about how to > > solve this without going > > > > to this method?. Manjula can you point out the > > changes we need in the > > > > parser. Then we can discuss the problem and > > available solutions. This > > > > will help us to avoid any problems occur in the > > future because of the > > > > changes. > > > > If we have two methods for xml reader creation, > > how we can use both of > > > > them in the Axis2/C engine. Are we going to check > > the MIME headers and > > > > create the required xml reader? > > > > > > In the MTOM case the serialized message is most > > probably in the > > > following format. > > > > > > --MimeBoundary > > > mime_header1:xxxxx > > > mime_header2:xxxxx > > > > > > <soapenv:Envelope>.......</soapenv:Envelope> > > > --MimeBoundary > > > mime_header1:xxxx > > > mime_header2:xxxx > > > > > > yyyyyyyyybinarydatayyyyyyyyyyyy > > > --MimeBoundary-- > > > > > > We have the information form http_headers whether > > this is a multipart > > > message or not. So will call the soap_builder > > straightaway with a flag > > > telling this is a multipart message. Since the > > modified soap_builder > > > will have a reference to mime_parser instance when > > seeing this flag it > > > will call the mime_parser. mime_parser will search > > for \r\n\r\n sequence > > > between SOAP part and first mime_headers set. When > > searching for this it > > > may read some part of the SOAP. So after this search > > we can't > > > straightaway call either xml_reader_create_for_io or > > > xml_reader_create_for_memory, because none of them > > will not get correct > > > xml. So what we need is first read from a buffer and > > then when that > > > buffer is finished read from the network stream. > > This will cause to > > > define and implement the new xml_reader function and > > corresponding > > > modifications at parser level as I mentioned in my > > previous mail. > > > > > > -Manjula. > > > > > > > > > > > > > > Thanks, > > > > Milinda... > > > > > > > > On Sat, Jul 12, 2008 at 6:53 PM, Manjula Peiris > > <[EMAIL PROTECTED]> > > > > wrote: > > > > Hi all, > > > > > > > > MTOM caching stuff is finished in both > > send and receiving > > > > sides.It is > > > > available here [1]. But due to this [2] > > problem, it cannot be > > > > integrated > > > > to Axis2/C trunk. > > > > > > > > In order to solve that problem we need to > > keep a reference of > > > > mime_parser instance in the soap_builder > > similarly > > > > stax_builder. So when > > > > parsing the soap part the xml_reader will > > not just read from > > > > the stream, > > > > but also need to read from the buffer > > which will be available > > > > after > > > > parsing mime_headrs. So we need to create > > a parser instance > > > > which will > > > > first read from the buffer and when that > > buffer is not > > > > containing any > > > > more data, it will read from the stream. > > So we may need to add > > > > a new > > > > function say, > > > > xml_reader_create_for_memory_and_io and > > change some of the > > > > logic in > > > > parser. > > > > Your suggestions are highly appreciated. > > > > > > > > > > [1] > https://svn.apache.org/repos/asf/webservices/axis2/branches/c/post_1_4_mtom/c > > > > > > > > > > [2] > http://marc.info/?l=axis-c-dev&amp;m=121320959601769&amp;w=2 > > > > > > > > Thanks, > > > > -Manjula. > > > > > > > > > > > > -- > > > > Manjula Peiris: > > http://manjula-peiris.blogspot.com/ > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > > > > > -- > > > > http://mpathirage.com > > > > http://wso2.org "Oxygen for Web Service > > Developers" > > > > http://wsaxc.blogspot.com "Web Services With > > Axis2/C" > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > > > > > > > > > > > -- > > Thilina Gunarathne - http://thilinag.blogspot.com > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Thilina Gunarathne - http://thilinag.blogspot.com
