On Fri, 2005-02-11 at 11:01, Aaron Stone wrote: > On Fri, Feb 11, 2005, Hans Kristian Rosbach <[EMAIL PROTECTED]> said: > > >> P.S.: If it was really meant to read everything into memory and then copy > >> the same message again to the insert query, why not consider reading > >> everything into a temp file, and then store the msg in db using something > >> like this: > >> > >> insert msg set id=xxx, body="first 512 bytes of msg goes here..."; > >> update msg set body=body+"another 512 bytes..." where id=xxx; > >> update msg set body=body+"yet another 512bytes" where id=xxx; > > > > Lets see.. our current max attachment size is (on our mailservers) > > 50MB, with 512bytes blocks this would lead to 102400 database queries. > > That would surely kill the database performance even under light load. > > Your suggestion of a maximum memory usage would be perfect here. Right now > we have no sanity checking against the maximum query size. That should be > added. Then we take the smaller of the two -- if the dbmail.conf > MAXMESSAGESIZE size is smaller, we just use that, but if the query size is > smaller, we use it and give an error message so that the sysadmin knows to > up this variable in the database. In any event, we should keep memory > usage below an admin specified limit and make sure never to give a query > that's too large to the database.
Absolutely. > [snip] > > 2. We can use a tempfile > > Now, this would lead to more database calls depending on block size > > and it would probably lead to worse performance. > > So wait -- the MTA takes the message, puts it in a file to queue for > delivery. Then the MTA reads the file either into a pipe or tcp connection > to dbmail. Then DBMail puts the message into a file to piece into the > database. So for every message, we put it on disk twice and into memory > three times... Well, actually the file can be opened with flags that hint to the OS that this is indeed a temporary file that might just be stored into memory cache and never even touch the disk (OS will decide depending on wether enough memory is free and how long it has lived in memory). > If DBMail became an MTA, then we could write messages to disk in CSV in > the first place, and use LOAD DATA INFILE (which is standard ANSI, I > believe) to stuff it into the database once we've done the delivery stuff. I for one hopes that dbmail will one day become an OPTIONAL simple MTA. The only special thing I need from it is support for antivirus and spam filtering. -HK
