> Manjula Peiris wrote: >> On Sun, 2008-03-16 at 16:26 +0530, Samisa Abeysinghe wrote: >> >> >>> We have an efficient parsing mechanism already, tested and proven to >>> work, with 1.3. Why on earth are we discussing this over and over >>> again? >>> >>> Does caching get affected by the mime parser logic? IMHO no. They are >>> two separate concerns, so ahy are we wasting time discussing parsing >>> while the problem at had is not parsing but caching? >>> >> >> No, the current implementation starts parsing after reading the whole >> stream. Because of that the parsing is simple and efficient. And for >> considerable size of large attachments(eg : 100MB) also it is working >> well. The only problem it has is the attachment size will depend on the >> available system memory. >> > > Still, my argument on the separation of concerns on caching vs. parsing > holds. > It is a question about what takes precedence over the other. If the > attachment is too large, we need to interleave the concepts, where you > read a considerable amount that is ideal in size in terms of caching, > parse it for MIME, and then cache it and move on.
Parsing will always be choice No. 1. We cache only if we can't handle it. However, the real issue is how are we going to implement "parse it for MIME, and then cache it and move on". I still think that it is better to stick to Thilina's viewpoint in having each attachment cached as a separate file. And, each attachment should be cached, even if it is small or large, when the content-length exceeds the threshold. This is because many small attachments == one big attachment. Thus, I'm still on the parse_1st->cache_1st->parse_2nd->cache_2nd->... approach. I don't think that a cache all at once will give us desirable results. >> >>> Writing the partially passed buffer was a solution to caching. Do we >>> have any other alternatives? If so what, in short, what are they? >>> >> >> We can keep current implementation and write the attachment to a file >> when it exceeds a certain threshold. This is inside mime_parser means at >> transport level. So we are not keeping the whole binary inside om_tree >> during the invocation of handlers and the receiver (may be the actual >> service or client) can straightaway access the file. Even though this >> approach will limit the attachment size we can handle to the system >> available memory , I think it has the added advantage of not keeping the >> attachment in memory. >> > > How does this compare with what I proposed above in this reply? > > Samisa... >> >>> Samisa... >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [EMAIL PROTECTED] >>> For additional commands, e-mail: [EMAIL PROTECTED] >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> >> > > > -- > Samisa Abeysinghe > Software Architect; WSO2 Inc. > > http://www.wso2.com/ - "Oxygenating the Web Service Platform." > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
