On Sat, 6 Sep 2003 00:08, Eric Sproul wrote: > On Fri, 2003-09-05 at 01:14, Russell Coker wrote: > > I was under the impression that Sendmail also queues everything to disk. > > How does it's queue operate then? > > While the message is coming in, Sendmail buffers the message to memory, > optionally piping the DATA portion to a socket (for milter scanning). > Only after the <CR>.<CR> does Sendmail accept responsibility for the > message (providing it was not rejected by a milter) and queue it. Some > might say this risky (power outages and such) but I would counter that
It's not risky, if anything goes wrong before the <CR>.<CR> the message must be discarded. However in a modern system there is no performance benefit in buffering to memory over writing to a file without sync(). For a large message you probably want to write it to disk instead of keeping it in memory to avoid thrashing. > until the entire message has been received and processed, the receiving > MTA is not responsible for the message. In fact, I think this is > RFC-specified. Why then, if the receiver isn't responsible, would it > want to spend disk I/O queuing a message that may end up being rejected > or may fail to come completely in? The incidence of messages that fail part way through is quite low. Expecially in communications between big servers (which corresponds to a large portion of the non-spam traffic). Optimising for the common case makes sense to me. Also I believe that in Ext3 if you write data to a file and then unlink the file before the data is committed to disk then the data will never be written. So there seems no loss as long as the file isn't opened with O_SYNC and you don't call fsync() (and no-one calls sync()). > > I'm not sure what the situation was like in 1999, now Qmail and LDAP > > support is adequate. > > But only with patches to the source code. And since it sounds like you > can't distribute modified binaries, you'd have to patch/build qmail on > every MTA. I choose not to install a development environment on my > production servers. I distribute only binary packages with apt from a > central repository. True, this is a significant issue, which is why I recommend Postfix. > > You need two mail storage servers for 60,000 accounts? > > Yes. Actually we now have 4 mail stores. We have discovered, at least > for our situation, that it is not wise to put more than 20K accounts on > a single mailstore. This is not so much for the mail delivery, but for > POP3. As many other ISP admins know, a large percentage of customers > are the psychotic kind, prone to POPing their multi-MB mailboxes every > $%^&[EMAIL PROTECTED] minute, and leaving all the messages on the server. This puts > a > non-trivial strain on even a fairly hefty dual-x86 box with H/W RAID5 > and 2GB of RAM. I have not noticed that. I have only noticed a very small portion of users doing that. With 1,000,000 users the number of psychotic POP users is small enough that you can deal with them individually. Maybe customers of Dutch ISPs are smarter than those of whichever country you are in. > Yes, I know we could set a larger minimum interval for POP, but the > political implications of generating tech support calls about "why can't > I POP my mail?" prevent it. Don't get me started on THAT. 8^o Why not change your POP server to instead of rejecting the connections to put gratuitous delays. So if the time since the last connection is < 5 minutes then make every operation take an extra 18 seconds (some pop servers have 20 second time outs). That would delay a minimal POP session by 72 seconds which better than halve the load. Also if you use Maildir format the impact of checking for mail should not be particularly high. The dentry cache is all that's consulted, give the server plenty of RAM and disk reads should be quite rare. > I did some more figuring on our mail volume and found that even though > each of our 4 mail routers processes 11-12 messages/second (each message > requires up to 20 LDAP lookups and a milter for spam filtering), I see A caching LDAP proxy would be good for this situation. Converting 20 LDAP lookups over the network to a single LDAP lookup and 19 accesses to a local cache daemon should provide some significant benefits. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/ Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]