On 3/30/2012 2:40 AM, Jeremie CEINTREY wrote:
Due to lack of any evidence I'll have to make some coarse educated
guesses here.
> I often encounter performance problems with postfix and mailing-list.
Mailing list servers. Implicates rapid and likely parallel delivery
over a short interval.
> I have a relay mail postfix filtering mail traffic for about 12 000
> mailboxes. Mailboxes are hosted on other mail servers.
>
> I use also amavisd-new with spammassassin and clamav.
> Postfix version is 2.8 and i use it with postscreen configuration.
This is likely not the problem.
> My problem is that I often see that mailq grows up to 7000 mails and it takes
> about 2 hours to deliver those mails. I don't understand why it is so slow.
> When i take a look at sender with qshape -s active i see who are sending
> mails and it is always mailling, but it is not spam traffic.
Let's assume you don't have any obvious Postfix mis-configuration.
So lets think this through logically, step by step. The first and
obvious issue is that mail is coming in faster than it can go out which
is why the queue is filling up. So we need to identify the cause of the
slow outbound queue. A couple of possible causes:
1. Downstream servers are limiting your delivery rate
2. You storage system doesn't have enough IOPS performance to handle
both the incoming write load *and* outgoing read load
#1 should be identifiable by lots of premature disconnects and/or 4xx
rejections from downstream servers. If this is indeed the problem you
can have admins of those servers make necessary changes, or you can
create individual relay transports configured with delays.
#2 can be fixed by replacing the queue disk with a faster device with
more IOPS capability, either a disk with higher RPM, an SSD, or multiple
disks in a striped RAID configuration, accomplished either with a
software RAID driver or with a hardware RAID controller.
Both #1 and #2 can usually easily be fixed by limiting the rate of
incoming mail, specifically by reducing the number of allowed parallel
connections. Both mailing lists and bulk mailers tend to send a large
volume of mail in a short period of time. The sending MTAs tend to do
this by opening multiple connections to your MTA, Postfix in this case,
which allows 50 connections per client by default. To limit the number
of inbound parallel connections, you would change the value of the
following parameter
smtpd_client_connection_count_limit
from the default of 50 to something much lower, such as 1. This will
limit the number of connections from mailing list servers, bulk mailers,
and spammers, while having no negative impact on normal inbound mail
flow. Regular/normal mail delivery is usually accomplished with one
email being sent over a single connection which is then closed upon
successful delivery. With 50 open connections, a sending MTA such as a
mailing list server, could potentially send hundreds of messages per
second. If your disk isn't fast enough, which is likely your problem,
it will spend all of its IOPS writing the inbound mail to the queue,
which takes precedence over reads, causing read starvation and thus slow
delivery. This is likely why your queue piles up and takes so long to
drain.
The net effect of this is that instead of your outbound queue piling up
messages, these messages pile up in the outbound queue of the sending
MTA, allowing you to receive them at a sane rate, and preventing your
queue from piling up, and delaying delivery.
An almost identical question came up on this list about a month or so
ago. Setting smtpd_client_connection_count_limit=1 solved the OP's
queue problem. It'll probably solve yours as well.
Hope this information is helpful. Please let us know if this fixes your
problem.
--
Stan
