Coming back to original example of a one-million message queue:
Postfix is designed to survive extreme overload, but all mail will
be delayed. This is no different than the road to the airport:
when it becomes full, all vehicles will be delayed. Both the Postfix
scheduler and the road to the airport have a finite capacity.
Once they become congested you get first-in, first-out.
If you want fairness with 1M+ messages and a scheduler with a fixed
memory budget, then you need a scheduler admission policy.
For the problem at hand, the important scheduler decisions are:
a) Output side: which (destination, recipients) to deliver next.
I think that this part does not need to be changed, precisely
because the scheduler can see only a subset of all recipients
in the mail queue. The trick is deciding what recipient subset
the scheduler gets to see.
b) Input side: how many recipients to read from a queue file.
This is an important part of Patrik's scheduler, but fairness
between multi- and single-recipient mail is not the issue here.
In the case of a queue full of single-recipient messages, the
only choice is to exclude a queue file from consideration (in
terms of the road to the airport, to not allow a car to enter
the road).
Basically this means some sort of "active queue" admission policy
that includes queue file skipping based on some policy.
A queue file can be skipped in more than one way. Besides moving a
file the deferred queue, a file can also be skipped by leaving it
in the incoming queue and setting the mtime time stamp a little
into the future. We could also add an "overflow" queue that is read
when the incoming queue becomes empty. Which option is better depends
on how large the amount excess of mail is.
Wietse
