On Thu, Mar 06, 2003 at 05:42:57PM -0500, Sam Varshavchik wrote: > >would be of making the low watermark so close to the high watermark. It > >would mean that the disk would be read more often, but how severely > >would it impact courier's performance? > > Depends on the disk subsystem.
That's a fair point, and has nothing to do with courier - any MTA would have similar issues. > >Has anyone else experimented with these values? And if so, what is > >your experience? > > > >Sam, do you have anything to add to this discussion based on your > >personal testing and experience? I am curious as to how you devised > >this algorithm. Is it a well-known algorithm, with documented tuning > >guidelines, or is it something you have devised yourself? > > It's my own invention. It's not much of an algorithm, really. The goal is > to avoid hitting the disk after every delivery attempt. That's it. > > When you have a large backup, something has to give. Perfect FIFO > scheduling either requires some sort of a disk-based database structure, > that orders messages by the next scheduled delivery attempt date, or to > keep everything in memory. > > The current algorithm avoids either unwanted alternative. The compromise > is that if the queue is dominated by a single domain not all available > delivery slots will be used. But as long as you've maxed out your network > bandwidth the additional pending deliveries won't get you anything anyway. > So all you need to make sure is that you're using all of your available > bandwidth. Well, in my case, not all my bandwidth was in use. The mail server sits at the centre of our network, and is a both a relay for outgoing mail, as well as the MX for 2 hosts, which I will call A and B (it then forwards to A and B with esmtproutes). Hosts A and B are connected via different pipes. Yesterday, we had 2 separate outages. At first, host A was down. Some its messages were in the queue. Then B went down, and its messages also began to pile up in the queue. Eventually, messages for A were evicted from the queue. And soon after, B came back up. And then A came up. So courier was busy delivering mail for B all the time, and didn't go back to disk to find messages for A until almost 4 hours later. It could have been longer. New incoming email would not let the low watermark go down. I had even considered restarting courier, to allow it to find those really old messages for A, but the situation was a learning experience, so I didn't. Bandwidth to B was maxed out, while bandwidth to A was negligible. > The other thing you've noticed is new, incoming mail, starving out the > backed up queue. This might be fixable by simply not queuing up new mail > while the in-memory queue is above the low watermark. Just queue it to > disk. When the in-memory queue hits the low watermark it'll just go to the > disk anyway. The danger here is that if the queue is above the low watermark, but all the messages in it are temporarily undeliverable, then all new mail will wait around on disk for a very long time, until the next restart of courierd. > But, the flip side of the coin is that some may want to give preferential > treatment to their primary destination mail domain when there's a large > queue backup. This might even be a tunable setting. I was thinking... how about some kind of timer instead of the low watermark? It would expire say every 5 or 10 minutes (a tunable parameter), and re-read the on-disk queue, giving a chance for courier to find those old messages. The re-reading of the disk queue could depend on time, rather than the number of messages. A high watermark would still exist, to limit the size of the in-memory queue cache. This would stop courier going back to disk after every delivery, and still make sure that old messages would not get neglected for too long. -- Anand Buddhdev http://anand.org ------------------------------------------------------- This SF.net email is sponsored by: Etnus, makers of TotalView, The debugger for complex code. Debugging C/C++ programs can leave you feeling lost and disoriented. TotalView can help you find your way. Available on major UNIX and Linux platforms. Try it free. www.etnus.com _______________________________________________ courier-users mailing list [EMAIL PROTECTED] Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
