On Wed, Jul 27, 2011 at 11:17:38AM -0400, Victor Duchovni wrote:
> > QSHAPE is one tool we were already using, and the good news is that
> > even during a send process (one of which is going on right now), the
> > active queue is generally very small. Like so:
> >
> > T 5 10 20 40 80 160 320 640 1280 1280+
> > TOTAL 9 9 0 0 0 0 0 0 0 0 0
> > yahoo.com 3 3 0 0 0 0 0 0 0 0 0
> > gmail.com 2 2 0 0 0 0 0 0 0 0 0
> > ymail.com 2 2 0 0 0 0 0 0 0 0 0
> > yahoo.in 1 1 0 0 0 0 0 0 0 0 0
> > hotmail.com 1 1 0 0 0 0 0 0 0 0 0
>
> Is this the active queue only, or active + incoming. If the latter
> (default qshape(1) output), then the output side of Postfix is not
> saturated, and you can send more mail by giving Postfix more work
> (increase the input concurrency, or decrease the input latency or both).
I should note that just because the active queue is nearly empty (which
is good) does not mean that the throughput is low, it just means that
the output capacity exceeds the input rate, which is what should be
case. You don't want to running on the cusp of congestive collapse
(what happens when the steady-state input rate exceeds output capacity).
Therefore, you really should measure the output rate, just a matter
of counting "qmgr[<pid>]: <queueid>: removed" messages in the logs,
and getting a messages/minute rate.
You get more fine-grained by measuring alpha-smoothed average delivery
latency, which is c+d in the delays=a/b/c/d log entries from postfix/smtp.
http://en.wikipedia.org/wiki/Exponential_smoothing
On high throughput systems, I tend to use an alpha value of 0.05 or
less, since a single large message (the median messages is around 10K,
the average is closer to 100K, and large messages are over 10MB), with a
long transmission delay, or the occasional timed-out connection, should
not substantially skew the estimated delivery latency.
I eliminate duplicates caused by multi-recipient mail, then alpha-smooth,
then output the smoothed value at the last message for each time-quantum
(when the new message minute is different from the previous message
minute, output the smoothed running average, then blend in the new message).
--
Viktor.
