RE: [Dbmail-dev] new features status

2003-07-08 Thread Jesse Norell
> > What other features are there that require schemea changes? I didn't > > really see any. > > Nothing else immediately comes to mind. Implimenting some of those > features (with any requisite schema changes), stablizing the code > thereafter and a few misc. bug fixes was pretty much the gam

RE: [Dbmail-dev] new cvs tag - combining database drivers

2003-07-08 Thread Jesse Norell
Hello, Just thought it an opportune time to mention - the most recent patch I submitted has been very stable. Ie. the one described: http://twister.fastxs.net/pipermail/dbmail-dev/2003-June/000171.html It should be safe to put in now, and then if things break we'll blame it on all the new C++ c

RE: [Dbmail-dev] Current db schema & future improvements

2003-07-08 Thread Jesse Norell
Hello, > 4) duplicate messages on insertion > This i don't see as a real option using checksums. It would be nice, > but probably no two messages are exactly the same - deliver time, > internal date and stuff will probably break the checksum giving only > minimal storage benefits with a lot of

[Dbmail-dev] new cvs tag - combining database drivers

2003-07-08 Thread
Hi, Today we tagged current cvs and started working on the combining of the postgresql and mysql driver code - the code is practically the same and maintaining the same stuff on different locations just isn't convenient. We've chosen to build the new driver system upon C++. No it's not beautif

[Dbmail-dev] Current db schema & future improvements

2003-07-08 Thread
Hi all, i've been a bit too busy lately so i was pleasantly surprised by all the new discussion started :-) We do have some plans about optimising the database structure; i'll share out current view with y'all. First off, i'd like to make clear the current dbmail database model: * message ge

[Dbmail-dev] Reminder

2003-07-08 Thread pareshshah
Just a reminder, there is a dbmail channel on freenode, irc.freenode.net channel #dbmail where a few of us hang out and discuss things, so feel free to stop by and lurk, or join in. -- Ryan Butler <[EMAIL PROTECTED]> ADI Internet Solutions

[Dbmail-dev] Reminder

2003-07-08 Thread pareshshah
Just a reminder, there is a dbmail channel on freenode, irc.freenode.net channel #dbmail where a few of us hang out and discuss things, so feel free to stop by and lurk, or join in. -- Ryan Butler <[EMAIL PROTECTED]> ADI Internet Solutions _

[Dbmail-dev] Reminder

2003-07-08 Thread Ryan Butler
Just a reminder, there is a dbmail channel on freenode, irc.freenode.net channel #dbmail where a few of us hang out and discuss things, so feel free to stop by and lurk, or join in. -- Ryan Butler <[EMAIL PROTECTED]> ADI Internet Solutions

Re: [Dbmail-dev] new features status

2003-07-08 Thread Jesse Norell
Hello, > BTW: I'm sure this is over simplistic, but is there a reason that we can't > just use triggers? I know postgres supports them, and the also have built > in regex, it would seem very easy to just add a trigger that says, when a > message inserted into the messages table, update the folde

Re: [Dbmail-dev] new features status

2003-07-08 Thread Matthew T. O'Connor
From: "Aaron Stone" <[EMAIL PROTECTED]> > PS - Let's clarify definitions of 'sorting' and 'filtering' since we have > two distinct stages where we want each of these to happen... > > Sorting: > Happens within DBMail. Users store a list of rules and patterns to > match against incoming mes

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Matthew T. O'Connor
From: "Magnus Sundberg" <[EMAIL PROTECTED]> > Matthew T. O'Connor wrote: > > This is why I suggested the value pair design, since we won't always > > know what's interesting. > > > Do you have any ideas of how to create an index, to get good > request performance? > something like > CREATE TABLE in

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Matthew T. O'Connor
From: "Jesse Norell" <[EMAIL PROTECTED]> > > Right - we're working on the dynamic caching because with us being > > too lazy to change dbmail to handle it itsself, that's the only > > practical option available. The only reason to add that to the pop3/ > > imap daemons would be for seemless migr

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Ryan Butler
On Tue, 2003-07-08 at 08:54, Matthew T. O'Connor wrote: > I have no idea, unfortunately, I don't really know how the imap clients > really work. > > My real question is what queries are being done by the server. When I > use evolution and ask it to search the message body for a piece of text,

RE: [Dbmail-dev] new features status

2003-07-08 Thread Aaron Stone
My status on the sorting work has moved back into the lower levels of the libSieve library. Some various and sundry work... reworking an address parser and gutting the API so that it is build around in-memory data rather than pulling things from disk and so that it returns linked lists of actions r

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Jesse Norell
> Right - we're working on the dynamic caching because with us being > too lazy to change dbmail to handle it itsself, that's the only > practical option available. The only reason to add that to the pop3/ > imap daemons would be for seemless migration (all new messages get > cached at injectio

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Jesse Norell
> Along with "what queries are being done by the server," make sure > we also look at whether these queries make sense. Ie. after looking > at common requests for IMAP clients, can the current queries done by > the server be improved upon? Another thing I forgot to mention; along with optimiz

[Dbmail-dev] db schema changes discussion /FLAGS

2003-07-08 Thread Magnus Sundberg
Hi, Is it optimal to store each flag in a separate byte? What about storing the flags as a bit in an int? This is a little bit more troublesome to handle on the database client side, but I think it offloads the server some. But there might be some SQL queries that require access to the flags

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Magnus Sundberg
Matthew T. O'Connor wrote: On Tue, 2003-07-08 at 11:18, Jesse Norell wrote: One other thought; I'm not recommending the approach for splitting message components (eg. file attachments) and saving as discrete components, as I think the work involved and complexity introduced may not be worth th

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Jesse Norell
> I believe this is currently the inefficient copy-everything > approach. If you move a message from one folder to another, I don't > know what it does, it may just update the mailbox_idnr in the > messages table entry. Search for shared folder discussions, as this > is planned on being addres

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Jesse Norell
> On Tue, 2003-07-08 at 11:18, Jesse Norell wrote: > > One other thought; I'm not recommending the approach for splitting > > message components (eg. file attachments) and saving as discrete > > components, as I think the work involved and complexity introduced > > may not be worth the benefits,

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Magnus Sundberg
Matthew T. O'Connor wrote: On Tue, 2003-07-08 at 06:44, Magnus Sundberg wrote: Hi again, I have done some more thinking about the database structure. I am not sure of the performance gain with a fast header table structure. This would be ideal with a single table and one VARCHAR column for ea

RE: [Dbmail-dev] new features status

2003-07-08 Thread Jesse Norell
Hello, Yes, dbmail-dev is a fairly recent addition - look at the dbmail list for anything older (well, even some of the subsequent postings there would be appropriate for dbmail-dev, too). > What is the status of the procmail-like filtering? I didn't know > procmail could do this. I would lov

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Matthew T. O'Connor
On Tue, 2003-07-08 at 11:18, Jesse Norell wrote: > One other thought; I'm not recommending the approach for splitting > message components (eg. file attachments) and saving as discrete > components, as I think the work involved and complexity introduced > may not be worth the benefits, but one of

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Jesse Norell
> > This would be ideal with a single table and one VARCHAR column > > for each _interesting_ header. How do we know what the clients > > think are interesting headers? > > This is why I suggested the value pair design, since we won't always > know what's interesting. Yep. We need to be able

[Dbmail-dev] new features status

2003-07-08 Thread Matthew T. O'Connor
In response to Jesse Norell's message a few days ago about new features requiring schemea changes, I started looking through the dbmail-dev archives (which I was surprised to find only go back to March 2003). So I have a few questions: What is the status of the procmail-like filtering? I didn't

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Matthew T. O'Connor
On Mon, 2003-07-07 at 12:59, Jesse Norell wrote: > > My vision here is a "fastheaders" table which has a message number and > > half a dozen columns defined. On some regular basis, or upon message > > insertion, the messageblks table is scanned and the first entry of each > > new message number, wh

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Matthew T. O'Connor
On Tue, 2003-07-08 at 06:44, Magnus Sundberg wrote: > Hi again, > I have done some more thinking about the database structure. > I am not sure of the performance gain with a fast header table > structure. > This would be ideal with a single table and one VARCHAR column > for each _interesting_ he

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Magnus Sundberg
Hi again, I have done some more thinking about the database structure. I am not sure of the performance gain with a fast header table structure. This would be ideal with a single table and one VARCHAR column for each _interesting_ header. How do we know what the clients think are interesting he

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Magnus Sundberg
Hello again, You know, you can turn on quite heavy logging for the mozilla imap client. This is just a short log from my surfing my inbox and sending a small mail to myself (That is copied to the sent folder). To get the imap commands use "grep SendData imaplog.txt" I am sorry, but I can not

Re: [Dbmail-dev] db schema changes discussion

2003-07-08 Thread Magnus Sundberg
Hello, I patched my imap server (uw :-() to log the requests from my mail client (Mozilla 1.2.1). /Magnus The mozilla client generated the FETCH requests below FETCH (FLAGS) FETCH (FLAGS) FETCH (UID RFC822.SIZE BODY[]) FETCH (UID RFC822.SIZE BODY[]) FETCH (UID RFC822.SIZE BODY[]) FETCH (BODYS