On 12.5.2013 0:49, Viktor Dukhovni wrote:
On Sat, May 11, 2013 at 06:33:22PM -0400, Wietse Venema wrote:
Even simpler: stop reading the deferred queue when more than N% of
the maximal number of recipient slots is from deferred mail.
This does not address Patrick's stated goal of avoiding process
saturation in the smtp transport by slow mail to bogus destinations.
(Similar to my 2001 analysis that motivated "relay" for inbound mail
and taught me the importance of recipient validation).
Rather, it adresses active limit exhaustion. The idea is perhaps
a good one anyway. Reserve some fraction of the active queue limits
for new mail, so that when enough deferred mail is in core, only new
mail is processed along with the already deferred mail.
I too agree that this one would be really nice to have.
Postfix already prefers new mail when the active queue is full (which is
usually already too late, though), so it shouldn't be too difficult to
change either.
A separate mechanism is still needed to avoid using all ~100 smtp
transport delivery processes for deferred mail. This means that
Patrick would need to think about whether the existing algorithm
can be extended to take limits on process allocation to deferred
mail into account.
I have really tried, but unless I separated the two internally
considerably, I always winded up with the deferred recipients somehow
affecting the normal recipients. There is so many memory limits to deal
with, and once you let the deferred recipients in-core, it's hard to get
rid of them. The "less input" is a solution here, but I am afraid it
might affect the "real" deferred mail adversely to be generally
recommended...
The fact that qmgr doesn't know how many delivery agents for each
transport are there doesn't help either. It only knows the
var_proc_limit, which is not good enough for this. I recall we have had
a discussion with Wietse long time about this, and IIRC we decided that
it is better if qmgr doesn't depend on that value at that time...
I sympathise with the concern about the internal cost, but if the
solution adds substantial user-visible complexity I contend that
it is pointless, and the users who need this (the sites that accept
subscriptions via HTTP, ...) can just create a multi-instance
config, it is simple enough to do.
Hmm, if the visible configuration is what bothers you, it would be
equally trivial to implement it so qmgr splits the transport only
internally, and to the outside world it looks like if there was only one
transport. But I considered this a worse solution as it would do
something behind the scenes without allowing to configure it properly...
Patrik
