Patrick Chemla put forth on 1/9/2010 11:07 AM: > Hi, > > I will try all your advises, but something still very strange for me: > > We see that postfix logs show that ehlo process is very slow through > postfix but very fast by hand. Even I have recorded through > tcpdump/WireShark and I can see that messages are sent very very very > quickly in about 1 second. > > But still messages are sent at a rate of a dozen in 10 seconds. That > means that messages are sent 1 by one. > > If connexion to qmail servers are slow, or if qmails are mis-parameted, > too slow or anything else, When I do netstat -apn |grep :25 I get only a > few connexions from postfix server to qmail servers. Even if DNS+EHLO > are slow, and more, because DNS+EHLO seem to be slow, why I don't see > hundreds TCP connexions ESTABLISHED ?
This behavior is likely a result of the connection cache: http://www.postfix.com/CONNECTION_CACHE_README.html If one has a large amount of mail destined for a single host, it is inefficient to open dozens or hundreds of TCP connections and SMTP connections due to the additional overhead of process/thread count and memory consumption. It is much more efficient to pipeline all the mail through a single connection. One can only pump so many bits down the wire between two hosts. If you can fill the pipe to near capacity with one TCP/SMTP stream, why open 100s of connections to do the same? I believe this is why you are not seeing dozens or hundreds of TCP connections. Postfix is intelligently designed to avoid this inefficiency. > I expected that postfix will deliver on 30 qmail servers at the same > time, and should manage hundreds parallel deliveries, hundreds parallel > connexions. Is there some parameter or some conception rule that refrain > him to do so? > > I expected that postfix will full up his own CPU/memory creating these > parallel delivery processes or/and will wait after the qmail servers, > but on all servers at the same time, on multiple connections to each one. > > Am I correct ? or I am dreaming of another mail transport package? > > Patrick As Victor and others have already stated: 1. In your previous configuration, you had multiple thousands of unique IP addresses (your customers) connecting directly to your 30 qmail servers to relay their mail. qmail performed fine with this configuration because no one qmail server was seeing thousands of delivery attempts per minute from any one single IP address. 2. In your current Postfix configuration, your qmail servers are seeing a single unique IP address attempting to send multiple thousands of messages per minute, and qmail is reacting with rate limiting countermeasures because of this. You need to figure out what settings in the qmail configuration are controlling this rate throttling and in what way. Once you find this and change it, you should see a dramatic improvement in Postfix's ability to quickly move the mail out of the queue to the 30 qmail servers, most likely using a single or only a few TCP connections to each qmail server. -- Stan
