Re: [IMAP] MessageRow and MessageFlags

2007-03-15 Thread Stefano Bagnara
robert burrell donkin ha scritto: On 3/14/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: Robert Burrell Donkin wrote: > i can understand why some people may want to store message data in a database > what i cannot understand is why JAMES insists that it *has* to be > stored in a database Uh,

Re: [IMAP] MessageRow and MessageFlags

2007-03-15 Thread robert burrell donkin
On 3/14/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: Robert Burrell Donkin wrote: > i can understand why some people may want to store message data in a database > what i cannot understand is why JAMES insists that it *has* to be > stored in a database Uh, it doesn't. We have multiple mails

Re: [IMAP] MessageRow and MessageFlags

2007-03-14 Thread Danny Angus
On 3/13/07, Andrew C. Oliver <[EMAIL PROTECTED]> wrote: The Geronimo boys started that but aborted, They started to recreate *all* of the API including the javax.mail stuff. I think it would be more sensible to just write the content handlers, assuming that this is where the performance issues

RE: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Noel J. Bergman
Robert Burrell Donkin wrote: > i can understand why some people may want to store message data in a database > what i cannot understand is why JAMES insists that it *has* to be > stored in a database Uh, it doesn't. We have multiple mailstores, only one of which is DB based. --- Noel

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Stefano Bagnara
robert burrell donkin ha scritto: On 3/13/07, Stefano Bagnara <[EMAIL PROTECTED]> wrote: robert burrell donkin ha scritto: > On 3/13/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: >> Robert Burrell Donkin wrote: >> This is orthogonal for having the blob be in a >> separate table. The latte

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread robert burrell donkin
On 3/13/07, Andrew C. Oliver <[EMAIL PROTECTED]> wrote: Danny Angus wrote: > On 3/13/07, robert burrell donkin <[EMAIL PROTECTED]> wrote: > >> reinforces my view that JAMES should only parse data into a JavaMail >> message when this is demanded by a processor > > Totally. > > and we do need to wr

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread robert burrell donkin
On 3/13/07, Stefano Bagnara <[EMAIL PROTECTED]> wrote: robert burrell donkin ha scritto: > On 3/13/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: >> Robert Burrell Donkin wrote: >> This is orthogonal for having the blob be in a >> separate table. The latter allows us improved DB performance

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Stefano Bagnara
robert burrell donkin ha scritto: On 3/13/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: Robert Burrell Donkin wrote: > Noel J. Bergman wrote: > > Robert Burrell Donkin wrote: > > > the more i look into the model, though, the more i wonder whether > > > achieving good performance won't require

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Stefano Bagnara
robert burrell donkin ha scritto: reinforces my view that JAMES should only parse data into a JavaMail message when this is demanded by a processor - robert This is what I tried to achieve with the latest MimeMessageWrapper and MimeMessageCopyOnWriteProxy. That said, even if we don't use an

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Andrew C. Oliver
Danny Angus wrote: On 3/13/07, robert burrell donkin <[EMAIL PROTECTED]> wrote: reinforces my view that JAMES should only parse data into a JavaMail message when this is demanded by a processor Totally. and we do need to write our own implementation rather than use the com.sun stuff which is

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Danny Angus
On 3/13/07, Danny Angus <[EMAIL PROTECTED]> wrote: P.S. I'm happy to support a fully normalised data model, but I would suggest that if we're going to do that we should make (document) a logical model first. There are some de-normalisations which can also reap benefits, such as removing LOB's (

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Danny Angus
On 3/13/07, robert burrell donkin <[EMAIL PROTECTED]> wrote: reinforces my view that JAMES should only parse data into a JavaMail message when this is demanded by a processor Totally. and we do need to write our own implementation rather than use the com.sun stuff which isn't really suited to

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Danny Angus
On 3/13/07, robert burrell donkin <[EMAIL PROTECTED]> wrote: but the main issue in this case is 1-1 relationships are tough for OR mapping layers to optimize. torque frequently performs unnecessary joins across to message_flags. ATM this results in tables scans which may be possible to index awa

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread robert burrell donkin
On 3/13/07, Andrew C. Oliver <[EMAIL PROTECTED]> wrote: snip de ja vu... > > That's a JavaMail issue, and needs to be eliminated. Regardless of whether > the message is in the DB or a file, we want to stream it if we are > processing it at all. This is orthogonal for having the blob be in a

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread robert burrell donkin
On 3/13/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: Robert Burrell Donkin wrote: > Noel J. Bergman wrote: > > Robert Burrell Donkin wrote: > > > the more i look into the model, though, the more i wonder whether > > > achieving good performance won't require changes > > It likely will. There

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Andrew C. Oliver
snip de ja vu... That's a JavaMail issue, and needs to be eliminated. Regardless of whether the message is in the DB or a file, we want to stream it if we are processing it at all. This is orthogonal for having the blob be in a separate table. The latter allows us improved DB performance

RE: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Noel J. Bergman
Robert Burrell Donkin wrote: > Noel J. Bergman wrote: > > Robert Burrell Donkin wrote: > > > the more i look into the model, though, the more i wonder whether > > > achieving good performance won't require changes > > It likely will. There is a change I want to make, which is to move the > > mess

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread robert burrell donkin
On 3/13/07, Noel J. Bergman <[EMAIL PROTECTED]> wrote: Robert Burrell Donkin wrote: > the more i look into the model, though, the more i wonder whether > achieving good performance won't require changes It likely will. There is a change I want to make, which is to move the message data (the bl

RE: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Noel J. Bergman
Robert Burrell Donkin wrote: > the more i look into the model, though, the more i wonder whether > achieving good performance won't require changes It likely will. There is a change I want to make, which is to move the message data (the blog) to a separate table. We discussed that in Dublin. We

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread robert burrell donkin
On 3/13/07, Danny Angus <[EMAIL PROTECTED]> wrote: I don;t know but.. I think it is because they were added later, and we generally like to avoid changing the datamodel in ways which aren't backwards compatible. yeh the more i look into the model, though, the more i wonder whether achieving go

Re: [IMAP] MessageRow and MessageFlags

2007-03-13 Thread Danny Angus
I don;t know but.. I think it is because they were added later, and we generally like to avoid changing the datamodel in ways which aren't backwards compatible. If we change the datamodel we should provide a migration script for upgrades, need to either release db alter-table & update scripts or

[IMAP] MessageRow and MessageFlags

2007-03-12 Thread robert burrell donkin
i've been experimenting with indexes recently. so, i've spending some time going though SQL logs looking for table scans. there are a lot of joins from message to message_flags which result in unnecessary full table scans. i think that message and message_flags are joined one-to-one. this is inef