On Wed, Jun 24, 2020 at 04:43:55PM -0400, Wietse Venema wrote:
> For a given transport and destination, if all three classes have
> work then round-robin interleaving gives each class gets 1/3 of the
> delivery slots, 1/2 if there are only two classes with work, all
> slots if there is only one class witth work, and zero if there is
> no work.
>
> If implemented in oqmgr, it's brain-dead round-robin with FIFO.
> What could possibly go wrong? :-) Well it does not interleave
> deliveries within the same class. But, with single-recipient messages,
> that wasn't happening anyway, so we are not making things worse.
My instinct is that just "oqmgr" + coarse classes is not quite enough.
There can be multiple large messages in one of the larger classes that
can take a while to drain to a popular destination. There should
still be some sort of preemption after a particular message group
has consumed enough resources.
Hence the idea of trying to group related single-recipient messages to
synthetically identify a batch of messages that should intermittently
yield to allow other deliveries.
One way to do that is to trim the sender localpart to drop the
recipient extension, but more generally to have a regexp map
that can be configured for that, which might keep only the
domain, or recognize specific expected bulk flows.
All message with the same sender-based key (plus perhaps quantised
size or some other coarse metric) would dynamically consistitude
a qmgr "job", but I don't know how hard it is, or whether it would
be a problem that'll be dynamically updating the job as new messages
arrive, whereas before some of the job info was determined once up
front.
--
Viktor.
