Markus Stumpf <[EMAIL PROTECTED]> wrote:
>The "main" work at concurrencyremote=500 was finished after about 1250
>seconds, at concurrencyremote=150 it was finished after about 1450
>seconds; concurrencyremote=250 is in between at about 1350 seconds.
I think that's because you're not just measuring the local system's
ability to pump out messages, but also the ability of an unknown set
up remote hosts to receive them.
>The number of finished successful deliveries/second is nearly the same
>for all three data sets (about 75-80 deliveries/second).
I think what you've determined is that the remote hosts on your list
can only accept 75-80 deliveries/second, not that qmail can't deliver
faster than that. In these tests, your system was able to keep 500
qmail-remotes busy, but it's possible that if the remote systems were
more receptive--able to receive 200, 300, 400 messages/second--it
might not have been able to keep them all busy.
>However the number of failures/deferrals per second was lower in the
>150 data set than in the 250 and much lower than in the 500.
Evidence that you're overwhelming some of the remotes. Do you see
deferrals for different reasons with higher concurrencies?
>*MY* conclusion from that comparisons is that the power of the
>qmail-bigconcurrency patch is probably commonly overestimated
>and the patch is kinda useless.
>
>PLEASE NOTE: the data sets are collected from delivery cycles of three
> successive weeks (the newsletter is a weekly one). Although it's
> delivered the same weekday (Friday) and around the same time (early
> afternoon GMT+2) the load on the remote (i.e. receiving) mail servers
> has a large impact on the data. This is even more true as 90% of
> the messages are sent to only 300 unique IP addresses (some of which
> are surely hidden behind load balancers).
> Thus minor tendencies are to be handled with care and the data sets
> may not be really representative.
I'm not sure you can safely draw that general conclusion. Clearly, in
this case it's nearly useless. But I don't think your situation is
typical--90% of the messages to 1/3% of the hosts. And 90000 is a
pretty large list. The chances are good that a large percentage of
those 300 hosts couldn't handle a sudden surge of up to 500 additional
SMTP sessions--or least they couldn't handle it well.
I think smaller mailings and/or a more diverse distribution of
recipients would reveal more impact of a higher concurrencyremote.
>I'd be very interested in your opinions/comments.
Thanks for taking the time to publish this information. It's further
demonstration of the need for profiling versus speculation. And it
points out how difficult it can be to accurately measure the impact of
a local change when the measurement depends on the response of remote
systems that have the potential to adapt (positively or negatively,
intentionally or accidentally) to the local change.
-Dave
