RE: stream-based MIME parser

2004-04-12 Thread Noel J. Bergman
> So that basically means, you're not dealing with a "modifiable" > stream, so much as dealing with a "copying" callback handler. Right. I see two broad classes of handlers: sinks and emitters. Sinks just eat the stream, whereas emitters output a possibly modified stream. > I guess the challeng

Re: stream-based MIME parser

2004-04-12 Thread Joe Cheng
For matchers those should be fine. For mailets, we would want to be able to process to a point, emit content for a new header or part, and then continue. I am imagining it as: MimeMessageSource -> MIME Parser -> Event Handler(s) -> MIME Stream If we want Mail to be re-writeable, we'd probably

RE: stream-based MIME parser

2004-04-12 Thread Noel J. Bergman
> Just want to make a note that my MIME parser currently presents you with > streams that are non-seekable, non-markable, and non-writable. Not sure > if anyone was expecting anything different, but there it is. For matchers those should be fine. For mailets, we would want to be able to process

Re: stream-based MIME parser

2004-04-12 Thread Joe Cheng
Just want to make a note that my MIME parser currently presents you with streams that are non-seekable, non-markable, and non-writable. Not sure if anyone was expecting anything different, but there it is. In other words, even if you've parsed through the body once and have somehow recorded th

Re: stream-based MIME parser

2004-04-12 Thread Serge Knystautas
Noel J. Bergman wrote: Finding extra help is always nice, but I worry about expanding our scope early on by including many projects' objectives. No interest in expanding scope. I don't see that the scope has changed since we started talking about it last year, and I don't believe that it would hav

RE: stream-based MIME parser

2004-04-12 Thread Noel J. Bergman
> I would prefer to develop it here since we have a very tangible problem > to solve. We're in agreement on both the approach and the reasons. > it might be worth looking at Alex's streaming codec, and > adopting is a common interface for moving the data into > the package. > Finding extra help

Re: stream-based MIME parser

2004-04-11 Thread Serge Knystautas
Noel J. Bergman wrote: The reason I mentioned Jakarta Commons was because of a thread back in March*. There has been some interest in Multipart MIME there. But I think it might be best to develop it here, and then consider moving it to Commons. Frankly, I don't want to be held to a coordinated sc

RE: stream-based MIME parser

2004-04-11 Thread Noel J. Bergman
Serge, The reason I mentioned Jakarta Commons was because of a thread back in March*. There has been some interest in Multipart MIME there. But I think it might be best to develop it here, and then consider moving it to Commons. Frankly, I don't want to be held to a coordinated schedule with Com

RE: stream-based MIME parser

2004-04-11 Thread Steve Brewin
Serge Knystautas wrote: > Noel J. Bergman wrote: > > The whole package is about 50K of code. We should probably > import this > > through the Incubator as a pass-through for James. I'm not > sure if it would > > be best to add it as org.apache.james.util.mime, and > perhaps eventually > > migrate

Re: stream-based MIME parser

2004-04-11 Thread Serge Knystautas
Noel J. Bergman wrote: To quote Joe: "It transparently decodes Base64 and quoted-printable on demand (i.e., doesn't base64-decode that 5MB binary attachment until you actually ask for its bytes). It successfully deals with the "torture-test.mbox" message posted by Mark Crispin: http://groups.googl

RE: Stream-based MIME Parser

2004-03-05 Thread Noel J. Bergman
> Are you saving those mails? :) Not intentionally at this time, although they would be in my error/ repository. It is all too easy to crash JavaMail with invalid content. When we are able to focus on fixing that, I'll start to worry about test cases. --- Noel

Re: Stream-based MIME Parser

2004-03-05 Thread Joe Cheng
Well, actually, I am getting too many exceptions from it with real-world mail, and in cases where I really should not need to care. Are you saving those mails? :) The world could really use a corpus of problematic MIME. (Assuming that they don't contain credit card numbers, missile launch

RE: Stream-based MIME Parser

2004-03-04 Thread Noel J. Bergman
> > What I believe we want is code that reads through the MIME content > > as a stream and issues callbacks for each header and part. For a > > part (counting the entire message as a part), I'd like to be able > > to get the raw stream and just read on it until the end of that > > scope. From that,

RE: Stream-based MIME Parser

2004-03-04 Thread Noel J. Bergman
> I know people aren't thrilled with the JavaMail parser, but the one > thing it does very very well (meaning it takes a lot of effort) is > spec implementation. Well, actually, I am getting too many exceptions from it with real-world mail, and in cases where I really should not need to care. > T

RE: Stream-based MIME Parser

2004-03-03 Thread Danny Angus
> OK. So, let me make sure I have this straight. There are MIME types > that are, to the parser, opaque--it doesn't actually need to be aware of > these types or how to handle them, it just needs to return the content > to the caller (handling the proper decoding if necessary). For example,

Re: Stream-based MIME Parser

2004-03-03 Thread Joe Cheng
Finally.. MIME is extensible through "official" mechanisms for sepcifying "official" new MIME types, make your parser extensible too and you can keep up to date. OK. So, let me make sure I have this straight. There are MIME types that are, to the parser, opaque--it doesn't actually need to b

Re: Stream-based MIME Parser

2004-03-03 Thread Serge Knystautas
Joe Cheng wrote: >> What I believe we want is code that reads through the MIME content as a stream and issues callbacks for each header and part. For a part (counting the entire message as a part), I'd like to be able to get the raw stream and just read on it until the end of that scope. From t

Re: Stream-based MIME Parser

2004-03-03 Thread Danny Angus
> except that many MIME producers seem not to > follow the spec. (Danny, any tips?) Tips.. MIME is ruthlessly logical and extremely fully specified, but it is basically straightforward enough, understand the spec before you start. MIME is a pretty simple idea which gives us powerful & complex

Re: Stream-based MIME Parser

2004-03-03 Thread Joe Cheng
>> What I believe we want is code that reads through the MIME content as a stream and issues callbacks for each header and part. For a part (counting the entire message as a part), I'd like to be able to get the raw stream and just read on it until the end of that scope. From that, we can do a

RE: Stream-based MIME Parser

2003-12-12 Thread Danny Angus
> Yes, that is the W3C code I was looking at, and you've a good point about > seeing if Tomcat, possibly Jetty, has similar code. commons fileupload has to deal with mime parsing too, and it returns (I think) headers+streams for each part of a multipart/* Mime parsers aren't rocket science,

RE: Stream-based MIME Parser

2003-12-11 Thread Noel J. Bergman
[EMAIL PROTECTED] wrote: > Noel J. Bergman wrote: > > What I really could use is a good stream-based MIME parser. I am starting > > to look at the W3C code and grendel code. > There is some MimeParser code under Jigsaw that is lightweight but > mostly ment for form encoding. There should be one u

Re: Stream-based MIME Parser

2003-12-11 Thread Serge Knystautas
Noel J. Bergman wrote: We have to get away from loading entire messages into the heap unless we are editing them. What I really could use is a good stream-based MIME parser. I am starting to look at the W3C code and grendel code. The latter appears to be JavaMail-ish, but the W3C code uses callba

Re: Stream-based MIME Parser

2003-12-11 Thread harmeet
- Original Message - From: "Noel J. Bergman" <[EMAIL PROTECTED]> > What I really could use is a good stream-based MIME parser. I am starting > to look at the W3C code and grendel code. Which w3c code are you looking at ? There is some MimeParser code under Jigsaw that is lightweight