On Sun, May 12, 2013 at 10:12:06PM -0400, Wietse Venema wrote:
> Viktor Dukhovni:
> > (the queue manager is the
> > most expensive to support with complex features).
>
> That's only because I haven't let anyone else hack around in the
> master daemon. Changing this code is incredibly expensive.
OK, master(8) is not trivial, and yet the busy/idle logic is
relatively simple. All we need is a new per-service counter (the
blocked count), and a timer event turned on by children entering
the "blocked" state and turned off by children leaving the blocked
state (including process termination).
No change is free, but I would like to suggest that your response
(if it is a negative gut reaction to my proposal, rather than a
simple observation) is a bit hasty.
The reasonable response to latency spikes is creating concurrency
spikes. This limits the maximum latency for the busy state (normal
mail) to ~5s (which is commensurate with normal delivery latency
to remote sites that do various DNS lookups, ...), which gives a
throughput throughput floor of process_limit/5 rather than
process_limit/30 (or possibly even higher denominators with additional
delays in DNS lookups and delays between connect and "MAIL").
If my suggestion is accepted we need to include the DNS resolution
delay in the "blocked" time. The DNS timeouts are controlled by
the DNS library and can be long on some systems (dominating connection
timeouts).
Please consider this option, I think it addresses the issue head-on.
It also avoids the creation destination concurrency spikes, which
may be frowned upon by remote destinations. With multiple transports
the concurrencies add up, with my proposal delivery goes on for
unblocked destinations but blocked destinations don't get increased
per-destination concurrency, rather the added concurrency is an
aggregate across many destinations.
--
Viktor.
