[Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread kbaker
So I've been discussing the merits of DBMail on this thread for the past few days. They have a couple users interested in building an IMAP server with an SQL back-end, so I thought I'd tern them onto DBMail. Anyway after much discussion, the below message was posted. Was wondering if you guys

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread M. J. [Mike] OBrien
it could happen. best... Mike - Original Message - From: "kbaker" <[EMAIL PROTECTED]> To: "DBMAIL Developers Mailinglist" Sent: Wednesday, June 14, 2006 7:49 PM Subject: [Dbmail-dev] Re: The Future of Email is SQL So I've been discussing the merits of DBM

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Aaron Stone
On Wed, 2006-06-14 at 16:49 -0700, kbaker wrote: > So I've been discussing the merits of DBMail on this thread for the past few > days. They have a couple users interested in building an IMAP server with an > SQL > back-end, so I thought I'd tern them onto DBMail. Timo's a good guy, and I think

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Geo Carncross
I think the general concern of this isn't an anonymous user injecting a message into a DBMail managed site via SMTP, but instead an IMAP user account that might be able to get code leaked directly to the SQL engine without being escaped. I too am worried about this- and haven't managed to do an au

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread M. J. [Mike] OBrien
With all due respect: 1) A rant with no specifics on known security exploits, may raise attention and awareness but doesn't offer much. 2) Please *Do not cry wolf* unless you see and unequivocally identify a wolf. I have not seen any specific security breaches with DBMail in some 5 years n

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread kbaker
M. J. [Mike] OBrien wrote: With all due respect: 1) A rant with no specifics on known security exploits, may raise attention and awareness but doesn't offer much. Mike/List sorry to start a fire storm on DBMail security. I am very happy with DBMail and its continued development. I just happe

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Geo Carncross
On Thu, 2006-06-15 at 13:45 -0400, M. J. [Mike] OBrien wrote: > With all due respect: > > 1) A rant with no specifics on known security exploits, may raise attention > and awareness but doesn't offer much. Agreed. > 2) Please *Do not cry wolf* unless you see and unequivocally identify a > wolf

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Marc Dirix
6) Always maintain a good back-up regimen for your database. I think asking users to solve security issues by having "good backups" is really ridiculous. As most of us understood, he meant server-side backup of the database, so if something does go wrong, the admin can restore the pre-s

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread M. J. [Mike] OBrien
Yup. Well said Geo! Well said Marc! Hey, Kevin, you did good. I had heard verbally from someone about the thread but failed to follow up, but you did. Nice. I had actually though it was a rehashing of years back, like a flashback. Apparently not. I will take a stab at a Wiki piece next week in

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Jim Nasby
On Jun 15, 2006, at 1:20 PM, Geo Carncross wrote: This might actually be easier- All three supported databases (in a way) support a concept of access controls. With MySQL and Pg, each "mailbox" could be given a separate table (unless per-row grant rights are possible), and then it'd be the SQ

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Geo Carncross
On Thu, 2006-06-15 at 20:52 +0200, Marc Dirix wrote: > > > > > >> 6) Always maintain a good back-up regimen for your database. > > > > I think asking users to solve security issues by having "good backups" > > is really ridiculous. > > > > As most of us understood, he meant server-side backup of t

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Geo Carncross
On Thu, 2006-06-15 at 15:01 -0500, Jim Nasby wrote: > On Jun 15, 2006, at 1:20 PM, Geo Carncross wrote: > > This might actually be easier- All three supported databases (in a > > way) > > support a concept of access controls. With MySQL and Pg, each > > "mailbox" > > could be given a separate t

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Jim C. Nasby
On Thu, Jun 15, 2006 at 04:04:55PM -0400, Geo Carncross wrote: > On Thu, 2006-06-15 at 15:01 -0500, Jim Nasby wrote: > > On Jun 15, 2006, at 1:20 PM, Geo Carncross wrote: > > > This might actually be easier- All three supported databases (in a > > > way) > > > support a concept of access controls

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-15 Thread Aaron Stone
On Thu, Jun 15, 2006, "Jim C. Nasby" <[EMAIL PROTECTED]> said: > Is there some reason why bound parameters aren't being used? Aside from > the security aspect, they also provide a speed improvement. It's a lot of work, and nobody has championed it. It could very well be done during 2.3 developmen

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Blake Mitchell
Replied to the previous post, but from my work address, oops. It appears to be caught in the typical DBMail list lag. Anyway, what I said was: >>> I'm a big fan of bound database parameters as well, aside from the already mentioned security and performance aspects, it just makes the code seem

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Aaron Stone
On Thu, Jun 15, 2006, Blake Mitchell <[EMAIL PROTECTED]> said: > I'm a big fan of bound database parameters as well, aside from the > already mentioned security and performance aspects, it just makes the > code seem cleaner somehow. Do all three of the database back ends > currently supported h

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Paul J Stevens
Jim C. Nasby wrote: > Is there some reason why bound parameters aren't being used? Aside from > the security aspect, they also provide a speed improvement. Prepared statements are planned for 2.3+. They will however not be usable in some cases. Most queries (90%) will be convertable. They are

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Aaron Stone
On Fri, 2006-06-16 at 12:37 +0200, Paul J Stevens wrote: > > Jim C. Nasby wrote: > > > Is there some reason why bound parameters aren't being used? Aside from > > the security aspect, they also provide a speed improvement. > > Prepared statements are planned for 2.3+. > > They will however not

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Jim C. Nasby
On Fri, Jun 16, 2006 at 12:37:47PM +0200, Paul J Stevens wrote: > > > Jim C. Nasby wrote: > > > Is there some reason why bound parameters aren't being used? Aside from > > the security aspect, they also provide a speed improvement. > > Prepared statements are planned for 2.3+. > > They will ho

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Aaron Stone
On Fri, 2006-06-16 at 12:12 -0500, Jim C. Nasby wrote: > Can you provide some examples? I can't think of any reason why any query > couldn't use bound parameters, though of course if they're highly > customized you won't see a performance benefit from them... Take a look at db_imap_utf7_like in d

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Blake Mitchell
Is there appreciable overhead in just preparing each statement as it's built and only using it once? This would still allow binding of the parameters, to avoid the quoting quagmire. Obviously there are performance gains in re-using prepared statements. But it seems to me that a good first step

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-16 Thread Paul J Stevens
Blake Mitchell wrote: > Is there appreciable overhead in just preparing each statement as it's > built and only using it once? I guess we pretty much have to assume the penalty might be a potential DOS. > This would still allow binding of the > parameters, to avoid the quoting quagmire. Obviousl

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-17 Thread Matthew T. O'Connor
Paul J Stevens wrote: Blake Mitchell wrote: Is there appreciable overhead in just preparing each statement as it's built and only using it once? I guess we pretty much have to assume the penalty might be a potential DOS. What penalty are we talking about here? The cost of preparrin

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-17 Thread Aaron Stone
On Sat, 2006-06-17 at 04:30 -0400, Matthew T. O'Connor wrote: > Paul J Stevens wrote: > > Blake Mitchell wrote: > > > >> Is there appreciable overhead in just preparing each statement as it's > >> built and only using it once? > >> > > I guess we pretty much have to assume the penalty migh

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-18 Thread Marc Dirix
The PostgreSQL people over at about oh-say 7.1 had a problem. Their backup tool could produce _bad_ backup files. The program would complete, exit 0 and everything, but the backup file couldn't be loaded. A few hours with awk and sed, it was all straightened out, but it would seem that I r

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-19 Thread Jim C. Nasby
On Sat, Jun 17, 2006 at 02:17:49AM -0700, Aaron Stone wrote: > On Sat, 2006-06-17 at 04:30 -0400, Matthew T. O'Connor wrote: > > Paul J Stevens wrote: > > > Blake Mitchell wrote: > > > > > >> Is there appreciable overhead in just preparing each statement as it's > > >> built and only using it on

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-19 Thread Aaron Stone
On Mon, 2006-06-19 at 12:01 -0500, Jim C. Nasby wrote: > On Sat, Jun 17, 2006 at 02:17:49AM -0700, Aaron Stone wrote: > > I suspect that none of us knows the actual cost of creating a prepared > > statement. There's no reason to believe that the planning costs more or > > less vs. simply issuing a

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-19 Thread Paul J Stevens
Aaron Stone wrote: >> In any case, I fail to see how this is an issue: if you're using >> dynamically generated code in performance critical areas of the server >> you have bigger things to worry about than the overhead of prepared >> statements. > > I'd argue that the sorting code is not performa

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-19 Thread Geo Carncross
On Mon, 2006-06-19 at 22:04 +0200, Paul J Stevens wrote: > Aaron Stone wrote: > >> In any case, I fail to see how this is an issue: if you're using > >> dynamically generated code in performance critical areas of the server > >> you have bigger things to worry about than the overhead of prepared >

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-20 Thread Aaron Stone
On Mon, Jun 19, 2006, Paul J Stevens <[EMAIL PROTECTED]> said: > Yep. We should be able to rig an api that will allow us to replace > queries one by one. But I'm working on sqlite atm. About to get it to > pass the unit-tests. Bit of a problem getting the udf code to work on > sqlite3, though. I'm

Re: [Dbmail-dev] Re: The Future of Email is SQL

2006-06-20 Thread Paul J Stevens
Geo Carncross wrote: > Just FYI: VACUUM is no longer required in sqlite3- the database can > AutoVacuum as necessary. Great. > > ... and I thought we were stopping the REGEX stuff to take advantage of > the indexes where possible? Yes, but I've just been porting dbsqlite.c as-is, and the reg

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-19 Thread Geo Carncross
On Sun, 2006-06-18 at 14:08 +0200, Marc Dirix wrote: > As every good administrator, don't change a working system unless you > have to, and > regularly check if things are still ok. Administrating is not a job > of only creating new applications, improving application or spending > your bosse

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-19 Thread Paul J Stevens
Geo Carncross wrote: > So what you're telling me is that you think only developers of dbmail > should be weighing in on the subject of dbmail's security? That would be the death of dbmail. I'm no security expert at all, and afaik neither is Aaron. I'm just a guy who ran across dbmail a couple of

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-20 Thread Marc Dirix
And neither is DBMail. You may misunderstand everything else I've said, but understand me here: I'm suggesting that DBMail make data integrity and security a focus. As is also my viewpoint, however a certain level of security should not be embedded into dbmail if it *can* be done with ex

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-20 Thread Geo Carncross
On Tue, 2006-06-20 at 00:53 +0200, Marc Dirix wrote: > It would be nice, although I understood from Paul sqlite backend > isn't quite production ready, I'm using it in production. It's going into flux to work with SQLite3 right now. > if you'd have the time to write an howto on the wiki. With a

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-20 Thread Paul J Stevens
Geo Carncross wrote: > SQL injection is only one kind of attack- if DBMail can be controlled by > other means, a user (perhaps: an anonymous one) might have access to > _all_ mail, and might have access to damage all mail. Amen. Like how many users have their dbmail.conf world-readable? Perhaps

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-20 Thread Geo Carncross
On Tue, 2006-06-20 at 09:51 +0200, Paul J Stevens wrote: > > Geo Carncross wrote: > > SQL injection is only one kind of attack- if DBMail can be controlled by > > other means, a user (perhaps: an anonymous one) might have access to > > _all_ mail, and might have access to damage all mail. > > Ame

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-21 Thread Jim C. Nasby
On Mon, Jun 19, 2006 at 08:02:02PM -0400, Geo Carncross wrote: > > The point is, security and data intergrity *is* very important. But > > it shouldn't be the focuspoint at places where a better sollution is > > found outside dbmail. I.E. the sql injection: From what I understood > > has post

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-21 Thread Geo Carncross
On Wed, 2006-06-21 at 12:59 -0500, Jim C. Nasby wrote: > On Mon, Jun 19, 2006 at 08:02:02PM -0400, Geo Carncross wrote: > > > The point is, security and data intergrity *is* very important. But > > > it shouldn't be the focuspoint at places where a better sollution is > > > found outside dbmail

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-22 Thread Jim C. Nasby
On Wed, Jun 21, 2006 at 04:41:19PM -0400, Geo Carncross wrote: > On Wed, 2006-06-21 at 12:59 -0500, Jim C. Nasby wrote: > > On Mon, Jun 19, 2006 at 08:02:02PM -0400, Geo Carncross wrote: > > > > The point is, security and data intergrity *is* very important. But > > > > it shouldn't be the focusp

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-22 Thread Aaron Stone
On Wed, 2006-06-21 at 16:53 -0500, Jim C. Nasby wrote: > Of course, it goes without saying that DBMail should be coded to reduce > the likelyhood of any possibility of an attack. Totally. And not to rain on this parade, but as I don't have enough spare cycles right now to read and fully digest th

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-22 Thread Geo Carncross
On Wed, 2006-06-21 at 16:53 -0500, Jim C. Nasby wrote: > One possibility would be to have the database handle all authentication > and requests via stored procs, and restricting the dbmail database > account to only having access to those stored procs. Or you could use > per-user authentication to

Re: [Dbmail-dev] Re: The Future of Email is SQL - getting off topic

2006-06-22 Thread Geo Carncross
On Wed, 2006-06-21 at 16:31 -0700, Aaron Stone wrote: > On Wed, 2006-06-21 at 16:53 -0500, Jim C. Nasby wrote: > > > Of course, it goes without saying that DBMail should be coded to reduce > > the likelyhood of any possibility of an attack. > > Totally. And not to rain on this parade, but as I do

PATCH INCLUDED Re: [Dbmail-dev] Re: The Future of Email is SQL (on to sqlite)

2006-06-20 Thread Geo Carncross
On Tue, 2006-06-20 at 09:45 +0200, Paul J Stevens wrote: > > Geo Carncross wrote: > > > Just FYI: VACUUM is no longer required in sqlite3- the database can > > AutoVacuum as necessary. > > Great. This change is in the DIFF. > > ... and I thought we were stopping the REGEX stuff to take advanta

Re: [Dbmail-dev] Re: The Future of Email is SQL - trying to get back on topic :)

2006-06-19 Thread Geo Carncross
On Mon, 2006-06-19 at 16:22 +0200, Paul J Stevens wrote: > Geo Carncross wrote: > > > So what you're telling me is that you think only developers of dbmail > > should be weighing in on the subject of dbmail's security? > > That would be the death of dbmail. > > I'm no security expert at all, and

Re: PATCH INCLUDED Re: [Dbmail-dev] Re: The Future of Email is SQL (on to sqlite)

2006-06-20 Thread Paul J Stevens
Geo Carncross wrote: > On Tue, 2006-06-20 at 09:45 +0200, Paul J Stevens wrote: >> Geo Carncross wrote: >> >>> Just FYI: VACUUM is no longer required in sqlite3- the database can >>> AutoVacuum as necessary. >> Great. > > This change is in the DIFF. > >>> ... and I thought we were stopping the

Re: PATCH INCLUDED Re: [Dbmail-dev] Re: The Future of Email is SQL (on to sqlite)

2006-06-20 Thread Paul J Stevens
Geo Carncross wrote: > On Tue, 2006-06-20 at 09:45 +0200, Paul J Stevens wrote: >> Geo Carncross wrote: >> >>> Just FYI: VACUUM is no longer required in sqlite3- the database can >>> AutoVacuum as necessary. >> Great. > > This change is in the DIFF. Geo, plz explain again why you want to use UD

Re: PATCH INCLUDED Re: [Dbmail-dev] Re: The Future of Email is SQL (on to sqlite)

2006-06-20 Thread Geo Carncross
On Tue, 2006-06-20 at 16:36 +0200, Paul J Stevens wrote: > > Geo Carncross wrote: > > On Tue, 2006-06-20 at 09:45 +0200, Paul J Stevens wrote: > >> Geo Carncross wrote: > >> > >>> Just FYI: VACUUM is no longer required in sqlite3- the database can > >>> AutoVacuum as necessary. > >> Great. > > >