Re : slow transport, master.cf and maxproc value

2011-03-07 Thread myrdhin bzh
Hello,


> I would suggest you grep your mail log for all outbound smtp connections to 
>these
> 10 IP addresses, and document all the recipient domains.  Then add all these
> domains to your slow transport map.  If you still get those 421 errors, 
>completely
> disable concurrency.  If you still get the errors, you probably can't fix the 
>problem
> until you upgrade to 2.5+ and have access to rate delay controls.
> 
> Hope this helps.

Ok. I'll try to make that.
Thank you,
-- 
Myrdhin,






Re: Re : Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-07 Thread Victor Duchovni
On Mon, Mar 07, 2011 at 11:55:34AM -0600, Stan Hoeppner wrote:

> > Unfortunately, the newer postfix rate delay controls are still based on
> > the recipient domain, not the MX destination.
> 
> Yeah.  He can absolutely fix his current problem with it.  But if he has
> to configure huge delays to accomplish it, this obviously creates yet
> more problems:  a backed up queue and mail not delivered in a timely manner.

There is a decent chance that once he moves to 2.5 or later, the proposed
feedback tuning and larger failure cohort count will solve the issue without
rate tuning. The reported issue with the destination is concurrency, not
message rate.

The OP's mandate to solve the issues with 2.1.5 is rather unfortunate. I'd
strongly consider upgrading the legacy systems to 2.7.2 or 2.8.1, after
getting a bit of familiarity with  the new release on a test machine.

-- 
Viktor.


Re: Re : Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-07 Thread Stan Hoeppner
Noel Jones put forth on 3/7/2011 7:37 AM:
> On 3/7/2011 7:21 AM, Stan Hoeppner wrote:

>> I would suggest you grep your mail log for all outbound smtp connections
>> to these 10 IP addresses, and document all the recipient domains.  Then
>> add all these domains to your slow transport map.  If you still get
>> those 421 errors, completely disable concurrency.
> 
> Nice explanation and reasonable advice.

Thanks Noel.

> 
>> If you still get the
>> errors, you probably can't fix the problem until you upgrade to 2.5+ and
>> have access to rate delay controls.
> 
> Unfortunately, the newer postfix rate delay controls are still based on
> the recipient domain, not the MX destination.

Yeah.  He can absolutely fix his current problem with it.  But if he has
to configure huge delays to accomplish it, this obviously creates yet
more problems:  a backed up queue and mail not delivered in a timely manner.

> Maybe some clever firewall rules could help.

I recall Wietse talking about implementing something a while back
specifically for this case.  IIRC it was something like doing an
smtp_bind_address type thing with a bunch smtp clients, each bound to a
different virtual IP all on a single NIC, and sending out all the IPs
round robin fashion to prevent "bad sender IP reputation".  My memory is
fuzzy on this.

In lieu of this, the best solution I can think of is going multi
instance, say 6 extra instances, 7 total.  The following is not a
complete step by step how-to but an overview of the basic steps required.

On the primary instance, remove all current restrictions you've put in
place WRT this Orange farm problem.  Bind 6 new additional IP addresses
as virtual NICs.  On the "primary" instance do smtp_bind_address to the
current existing IP address, and the same for inet_interfaces.  Do this
for each of the 6 new instances, so all 7 Postfix instances only use one
IP address each.

On each of the 6 new instances, leave smtp_connection_cache_on_demand
enabled, and set smtp_destination_concurrency_limit to 18.  This will
allow you to have 18 concurrent smtp connections for each of 10 (primary
Postfix instance) active queue destination domains hosted by the Orange
farm, 180 maximum smtp connections into the Orange farm.

Setup a round robin DNS A record, say "outboundlds.yourdomain.tld"
pointing to these 6 new IP addresses.  At the edge firewall limit the 6
new IPs to outbound TCP 25 traffic only.  Configure each of the 6 new
Postfix instances as minimally as possible, and to only accept
connections from the IP of the primary instance.  On the "primary"
instance, you will have a transport_map containing all domains for which
you've experienced concurrent delivery problems.

orange.fr   smtp:outboundlds.yourdomain.tld
wanadoo.fr  smtp:outboundlds.yourdomain.tld
... smtp:outboundlds.yourdomain.tld
... smtp:outboundlds.yourdomain.tld
... smtp:outboundlds.yourdomain.tld

With this setup, you now have essentially an outbound farm of 6 Postfix
servers sending from 6 IP addresses to picky receivers.  As connection
concurrency checking is typically performed by smtpd's on an smtp client
IP address basis, and as each of those 10 Orange servers will allow 3
connections per IP address, you will now be able to have 18 concurrent
smtp connections to each of the 10 servers in that farm, or 180 total
concurrent connections.  Currently you can get 30 total connections
being they start flogging you.

I'd think a peak capacity of 180 concurrent connections should be enough
to fix your problem.

> Hmm, I'll wonder out loud if a check_recipient_mx_access table that
> returns FILTER slow: would help.  It would affect all recipients of a
> multi-recipient message, but maybe that would be acceptable.

I like my idea better.  ;)  But it is a kludge, and much more complex to
implement.  I've never done it, so it's possible what I've stated above
won't work, and was a big waste of my time combing through postconf5 and
Googling. :(

It sure would be nice if Postfix would simply round robin outbound smtp
connections across all inet_interfaces by default, or have a main.cf
option allowing an OP to enable such a thing easily.  If such a thing is
already in postconf5 I couldn't find it.  Using multiple instances to
simply accomplish outbound connection balancing across multiple
interface IPs is overkill.

-- 
Stan



Re: Re : Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-07 Thread Noel Jones

On 3/7/2011 7:21 AM, Stan Hoeppner wrote:

myrdhin bzh put forth on 3/7/2011 1:53 AM:


No, it's not SPAM.


Ok, agreed, not spam.


In fact, zeDomain.tld is a french know domain : wanadoo.fr (and orange.fr). :(


The same 10 servers handle mail for both wanadoo.fr and orange.fr.  All
Postfix controls relevant to your issue available in 2.1.5 are based on
connection concurrency, not rate limiting.  The connection concurrency
is on a *per domain* basis.  Orange is the largest ISP in France, yes?

How many other recipient domains you send mail to are also hosted by
this same Orange 10 server mail farm?  If your answer is "a lot", you
could completely disable concurrency and still possibly trip their 3
connections per server limit, simply due to the number of recipient
domains for which you have mail queued.  They didn't list them in the
same order, but if you sort them you'll see it's the same 10 servers.

smtp.wanadoo.fr.600 IN  A   80.12.242.62
smtp.wanadoo.fr.600 IN  A   80.12.242.148
smtp.wanadoo.fr.600 IN  A   193.252.22.65
smtp.wanadoo.fr.600 IN  A   193.252.23.67
smtp.wanadoo.fr.600 IN  A   80.12.242.9
smtp.wanadoo.fr.600 IN  A   80.12.242.53
smtp.wanadoo.fr.600 IN  A   80.12.242.142
smtp.wanadoo.fr.600 IN  A   80.12.242.82
smtp.wanadoo.fr.600 IN  A   193.252.22.92
smtp.wanadoo.fr.600 IN  A   80.12.242.15

smtp-in.orange.fr.  600 IN  A   80.12.242.148
smtp-in.orange.fr.  600 IN  A   80.12.242.53
smtp-in.orange.fr.  600 IN  A   80.12.242.9
smtp-in.orange.fr.  600 IN  A   193.252.23.67
smtp-in.orange.fr.  600 IN  A   193.252.22.65
smtp-in.orange.fr.  600 IN  A   80.12.242.142
smtp-in.orange.fr.  600 IN  A   80.12.242.62
smtp-in.orange.fr.  600 IN  A   80.12.242.82
smtp-in.orange.fr.  600 IN  A   193.252.22.92
smtp-in.orange.fr.  600 IN  A   80.12.242.15

I would suggest you grep your mail log for all outbound smtp connections
to these 10 IP addresses, and document all the recipient domains.  Then
add all these domains to your slow transport map.  If you still get
those 421 errors, completely disable concurrency.


Nice explanation and reasonable advice.



If you still get the
errors, you probably can't fix the problem until you upgrade to 2.5+ and
have access to rate delay controls.


Unfortunately, the newer postfix rate delay controls are still 
based on the recipient domain, not the MX destination.


Maybe some clever firewall rules could help.

Hmm, I'll wonder out loud if a check_recipient_mx_access table 
that returns FILTER slow: would help.  It would affect all 
recipients of a multi-recipient message, but maybe that would 
be acceptable.


  -- Noel Jones


Re: Re : Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-07 Thread Stan Hoeppner
myrdhin bzh put forth on 3/7/2011 1:53 AM:

> No, it's not SPAM.

Ok, agreed, not spam.

> In fact, zeDomain.tld is a french know domain : wanadoo.fr (and orange.fr). 
> :( 

The same 10 servers handle mail for both wanadoo.fr and orange.fr.  All
Postfix controls relevant to your issue available in 2.1.5 are based on
connection concurrency, not rate limiting.  The connection concurrency
is on a *per domain* basis.  Orange is the largest ISP in France, yes?

How many other recipient domains you send mail to are also hosted by
this same Orange 10 server mail farm?  If your answer is "a lot", you
could completely disable concurrency and still possibly trip their 3
connections per server limit, simply due to the number of recipient
domains for which you have mail queued.  They didn't list them in the
same order, but if you sort them you'll see it's the same 10 servers.

smtp.wanadoo.fr.600 IN  A   80.12.242.62
smtp.wanadoo.fr.600 IN  A   80.12.242.148
smtp.wanadoo.fr.600 IN  A   193.252.22.65
smtp.wanadoo.fr.600 IN  A   193.252.23.67
smtp.wanadoo.fr.600 IN  A   80.12.242.9
smtp.wanadoo.fr.600 IN  A   80.12.242.53
smtp.wanadoo.fr.600 IN  A   80.12.242.142
smtp.wanadoo.fr.600 IN  A   80.12.242.82
smtp.wanadoo.fr.600 IN  A   193.252.22.92
smtp.wanadoo.fr.600 IN  A   80.12.242.15

smtp-in.orange.fr.  600 IN  A   80.12.242.148
smtp-in.orange.fr.  600 IN  A   80.12.242.53
smtp-in.orange.fr.  600 IN  A   80.12.242.9
smtp-in.orange.fr.  600 IN  A   193.252.23.67
smtp-in.orange.fr.  600 IN  A   193.252.22.65
smtp-in.orange.fr.  600 IN  A   80.12.242.142
smtp-in.orange.fr.  600 IN  A   80.12.242.62
smtp-in.orange.fr.  600 IN  A   80.12.242.82
smtp-in.orange.fr.  600 IN  A   193.252.22.92
smtp-in.orange.fr.  600 IN  A   80.12.242.15

I would suggest you grep your mail log for all outbound smtp connections
to these 10 IP addresses, and document all the recipient domains.  Then
add all these domains to your slow transport map.  If you still get
those 421 errors, completely disable concurrency.  If you still get the
errors, you probably can't fix the problem until you upgrade to 2.5+ and
have access to rate delay controls.

Hope this helps.

-- 
Stan


Re : Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread myrdhin bzh
Hello,


> From: Stan Hoeppner
> 
> My spammer RADAR is beeping...

:)

No, it's not SPAM.

My client have a domain (for example clientDomain.tld) and a SMTP server. This 
server rewrite all email addresses from firstname.surn...@clientdomain.tld to 
firstname.surname-clientdomaine@zedomain.tld (zeDomain.tld is the 
problem...). My client have some internal lists too. When he send a mail to one 
of his internal list, for example servi...@clientdomain.tld, this mail is sent 
to fn1.n-clientdomaine@zedomain.tld, fn2.n-clientdomaine@zedomain.tld, 
etc. Sometime, the message :

smtp.zeDomain.tld[xxx.xxx.xxx.xxx] refused to talk to me: 421 mwinf5c20 ME Trop 
de connexions, veuillez verifier votre configuration. Too many connections, 
slow 

down. OFR004_104 [104]


In fact, zeDomain.tld is a french know domain : wanadoo.fr (and orange.fr). :( 
Wanadoo has changed his configuration last december 2010 and a lot of problem 
appears (cf in french 
http://entraide.orange.fr/assistance/messages/index/35320/messagerie-plus-aucun-email-ne-rentre-chez-107-000-utilisateurs-orange-fr.html?dub=2).



I'm not the only one who have this problem but orange/wanadoo not responding...
-- 
Myrdhin,





Re: Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread Stan Hoeppner
myrdhin bzh put forth on 3/6/2011 5:33 PM:
> Really, thank you for your patience :) I have difficulty understanding 
> perfectly 
> English...
> 
> 
> 
>> No, the *process* limit of "3" is implemented in master(8), which spawns 
>> processes on demand, up to the process limit. The concurrency limit 
>> (parallel 
>> deliveries to a single destination domain) is enforced in the queue-manager. 
>> The 
>> queue manager concurrency limit is a maximum, the actual concurrency will be 
>> lower when the master(8) daemon process limit is reached, or mail arrival is 
>> not 
>> high enough to reach peak concurrency.
> 
> So in my conf :
> 
> + transport :
>   zedomain.tldslow:
> 
> + main.cf :
>   transport_maps = hash:/etc/postfix/transport
>   slow_destination_concurrency_limit = 2
> 
> + master.cf :
>   slowunix  -  -  n  -  3 smtp -o smtp_connection_cache_on_demand=no
> 
> 
> Can i replace the '3' with a '-' like this :
>   slowunix  -  -  n  -  - smtp -o smtp_connection_cache_on_demand=no
> 
> I'm really confused about this configuration... I can't find the right 
> optimized 
> configuration...
> 
> To know DNS informations  about zedomain.tld SMTP servers, I executed this 
> command : 
> 
>   user# dig smtp.zedomain.tld A
> 
> I have 10 responses (round robin) :
>   ;; ANSWER SECTION:
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x01
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x02
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x03
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x04
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x05
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x06
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x07
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x08
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x09
>   smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x10
> 
> I know (and i tested) each smtp.zedomain.tld servers can only accept 3 max 
> concurrency connections : 3 for the xxx.xxx.xxx.x01, 3 for xxx.xxx.xxx.x02, 
> etc. 
> So I could make up to 30 concurrency connections but i would like to force 3 
> per 
> servers.
> 
> If i configure like this :
>   + main.cf :  slow_destination_concurrency_limit = 30
>   + master.cf :  slowunix  -  -  n  -  - smtp -o 
> smtp_connection_cache_on_demand=no
> 
> is it good ? If i have 22 mails for zedomain.tld, 'smtp's should send mails 
> to 
> the smtp.zedomain.tld in this order (with the 
> smtp_connection_cache_on_demand=no 
> option):

My spammer RADAR is beeping...

myrdhin bzh, what is your motivation here?  It appears you are trying to
setup your single Postfix server to push the maximum amount of mail per
second into some number of undisclosed domains that have multiple MX
hosts, by getting around their per MX host restrictions.  This isn't
something white hat mail OPs typically do.

If you have a large volume of legit mail for a single domain, and that
domain is limiting your delivery rate, causing _problems_ for your
systems, you should contact the mail OPs at that domain(s) and _ask_
them how best to address your problem.  Attempting to circumvent their
countermeasures is a good way to get yourself permanently blacklisted by
them.

-- 
Stan


Re : Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread myrdhin bzh
Really, thank you for your patience :) I have difficulty understanding 
perfectly 
English...



> No, the *process* limit of "3" is implemented in master(8), which spawns 
>processes on demand, up to the process limit. The concurrency limit (parallel 
>deliveries to a single destination domain) is enforced in the queue-manager. 
>The 
>queue manager concurrency limit is a maximum, the actual concurrency will be 
>lower when the master(8) daemon process limit is reached, or mail arrival is 
>not 
>high enough to reach peak concurrency.

So in my conf :

+ transport :
  zedomain.tldslow:

+ main.cf :
  transport_maps = hash:/etc/postfix/transport
  slow_destination_concurrency_limit = 2

+ master.cf :
  slowunix  -  -  n  -  3 smtp -o smtp_connection_cache_on_demand=no


Can i replace the '3' with a '-' like this :
  slowunix  -  -  n  -  - smtp -o smtp_connection_cache_on_demand=no

I'm really confused about this configuration... I can't find the right 
optimized 
configuration...

To know DNS informations  about zedomain.tld SMTP servers, I executed this 
command : 

  user# dig smtp.zedomain.tld A

I have 10 responses (round robin) :
  ;; ANSWER SECTION:
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x01
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x02
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x03
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x04
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x05
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x06
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x07
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x08
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x09
  smtp.zedomain.tld.471 IN  A   xxx.xxx.xxx.x10

I know (and i tested) each smtp.zedomain.tld servers can only accept 3 max 
concurrency connections : 3 for the xxx.xxx.xxx.x01, 3 for xxx.xxx.xxx.x02, 
etc. 
So I could make up to 30 concurrency connections but i would like to force 3 
per 
servers.

If i configure like this :
  + main.cf :  slow_destination_concurrency_limit = 30
  + master.cf :  slowunix  -  -  n  -  - smtp -o 
smtp_connection_cache_on_demand=no

is it good ? If i have 22 mails for zedomain.tld, 'smtp's should send mails to 
the smtp.zedomain.tld in this order (with the 
smtp_connection_cache_on_demand=no 
option):
mail #01 -> xxx.xxx.xxx.x01
mail #02 -> xxx.xxx.xxx.x02
mail #03 -> xxx.xxx.xxx.x03
mail #04 -> xxx.xxx.xxx.x04
mail #05 -> xxx.xxx.xxx.x05
mail #06 -> xxx.xxx.xxx.x06
mail #07 -> xxx.xxx.xxx.x07
mail #08 -> xxx.xxx.xxx.x08
mail #09 -> xxx.xxx.xxx.x09
mail #10 -> xxx.xxx.xxx.x10

mail #11 -> xxx.xxx.xxx.x01
mail #12 -> xxx.xxx.xxx.x02
mail #13 -> xxx.xxx.xxx.x03
mail #14 -> xxx.xxx.xxx.x04
mail #15 -> xxx.xxx.xxx.x05
mail #16 -> xxx.xxx.xxx.x06
mail #17 -> xxx.xxx.xxx.x07
mail #18 -> xxx.xxx.xxx.x08
mail #19 -> xxx.xxx.xxx.x09
mail #20 -> xxx.xxx.xxx.x10

mail #21 -> xxx.xxx.xxx.x01
mail #22 -> xxx.xxx.xxx.x02

???
Thanks
-- 
Myrdhin,






Re: Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread Victor Duchovni
On Sun, Mar 06, 2011 at 10:43:54PM +, myrdhin bzh wrote:

> >   slow  unix  -   -   n   -   3   smtp -o 
> >smtp_connection_cache_on_demand=no -o smtp_destination_concurrency_limit=2
> 
> >> The second "-o ..." option is pointless and should be removed.
> 
> Ok.
> 
> >> Concurrency limits are enforced in the queue manager, individual delivery 
> >>agents just deliver one message at a time, it is the queue manager that 
> >>knows 
> >>about multiple deliveries in progress.
> 
> So, in my "slow" configuration, the queue manager can only max "create"
> 3 'smtp' delivery agents.

No, the *process* limit of "3" is implemented in master(8), which spawns
processes on demand, up to the process limit. The concurrency limit
(parallel deliveries to a single destination domain) is enforced in the
queue-manager. The queue manager concurrency limit is a maximum, the actual
concurrency will be lower when the master(8) daemon process limit is reached,
or mail arrival is not high enough to reach peak concurrency.

> But each can only deliver one message at a time and i have only 
> one domain in my transport table to the 'slow' deliver, so only 2 'smtp' 
> delivery agents can be created and deliver concurrency. The 3 in my master.cf 
> slow line is useless ?

No, the "-o smtp_destination_concurrency_limit=2" is useless.

-- 
Viktor.


Re : Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread myrdhin bzh
Thanks Victor,


>> You really should be using 2.7.2 or later.


I would like to use this version now :) But i can't : it will be possible after 
servers migrating (waiting 1 month :( ...)


>   slow  unix  -   -   n   -   3   smtp -o 
>smtp_connection_cache_on_demand=no -o smtp_destination_concurrency_limit=2

>> The second "-o ..." option is pointless and should be removed.

Ok.


>> Concurrency limits are enforced in the queue manager, individual delivery 
>>agents just deliver one message at a time, it is the queue manager that knows 
>>about multiple deliveries in progress.

Excuse me, I have a little trouble understanding :-s

So, in my "slow" configuration, the queue manager can only max "create" 3 
'smtp' 
delivery agents. But each can only deliver one message at a time and i have 
only 
one domain in my transport table to the 'slow' deliver, so only 2 'smtp' 
delivery agents can be created and deliver concurrency. The 3 in my master.cf 
slow line is useless ?

Thank you for your help,
-- 
Myrdhin,






Re: Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread Victor Duchovni
On Sun, Mar 06, 2011 at 04:21:44PM +, myrdhin bzh wrote:

> > From : myrdhin
> > Thank you for your help. I'll try your solution.
> 
> Sorry, but i always have "Too many connections, slow down." in my 
> /var/log/mail.log.
> 
> My Postfix is old (mail_version = 2.1.5) butI am constrained to correct this 
> problem before doing the migration on another server (with a recent
> version of postfix...).

The scheduler features that you are new with Postfix 2.5 and required at
least Postfix 2.5.7. You really should be using 2.7.2 or later.

> Finally i do that :
> 
>   # in /etc/postfix/main.cf :
>   transport_maps = hash:/etc/postfix/transport
>   slow_destination_concurrency_limit = 2
> 
>   # in /etc/transport/transport :
>   zeDomain.tld slow:
> 
>   # in /etc/postfix/master.cf :
>   slow  unix  -   -   n   -   3   smtp -o 
> smtp_connection_cache_on_demand=no -o smtp_destination_concurrency_limit=2

The second "-o ..." option is pointless and should be removed. Concurrency
limits are enforced in the queue manager, individual delivery agents just
deliver one message at a time, it is the queue manager that knows about
multiple deliveries in progress.

-- 
Viktor.


Re : Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-06 Thread myrdhin bzh
Hello :),

> From : myrdhin
> Thank you for your help. I'll try your solution.

Sorry, but i always have "Too many connections, slow down." in my 
/var/log/mail.log.

My Postfix is old (mail_version = 2.1.5) butI am constrained to correct this 
problem before doing the migration on another server (with a recent version of 
postfix...).

Finally i do that :

  # in /etc/postfix/main.cf :
  transport_maps = hash:/etc/postfix/transport
  slow_destination_concurrency_limit = 2

  # in /etc/transport/transport :
  zeDomain.tld slow:

  # in /etc/postfix/master.cf :
  slow  unix  -   -   n   -   3   smtp -o 
smtp_connection_cache_on_demand=no -o smtp_destination_concurrency_limit=2


It seems to work : no "Too many connections, slow down." messages in my log 
file 
since 2 days but i'm afraid about a mail delivery slowdown...

I understand the "smtp_connection_cache_on_demand=no" option but i don't know 
if 
the "*_destination_concurrency_limit" option is per process of smtp program.

With my master.cf, can i make 6 concurrency connections max on zeDomain.tld ? 
Or 
is it only 2 max ?

Thanks for your help,
-- 
Myrdhin,






Re : Re : Re : slow transport, master.cf and maxproc value

2011-03-02 Thread myrdhin bzh
Thank you for your help. I'll try your solution.
-- 
Myrdhin,


 


Re: Re : Re : slow transport, master.cf and maxproc value

2011-03-02 Thread Victor Duchovni
On Wed, Mar 02, 2011 at 10:58:43PM +, myrdhin bzh wrote:

> >> # in master.cf :
> 
> >> slow  unix  -   -   n   -   5   smtp
> 
> > Something like that. The "5" process limit will mean that with multiple
> > "slow" domains at most 5 delivery processes will be used in total for
> > all "slow" destinations. This may not be what you want. It can however
> > reduce instances of a single MX host getting many connections when it
> > serves multiple slow domains. The downside is a potential bottleneck.
> 
> My /etc/postfix/transport file contains one domain (domain.tld). This domain 
> have 10 SMTP servers (smtp.domain.tld with round-robin DNS configuration). I 
> know that each SMTP server can only accept 3 concurrency connections.
> 
> I find lot of lines like this in my mail.log :
> 
> Mar  2 20:31:04 localhost postfix/smtp[5394]: 2A5529C4429: host 
> smtp.domain.tld[xxx.xxx.xxx.xxx] refused to talk to me: 421 mwinf5c20 ME Trop 
> de 
> connexions, veuillez verifier votre configuration. Too many connections, slow 
> down. OFR004_104 [104]
> 
> How can i configure my postfix ?

Instead of setting a very low concurrency limit (the default is 20
which should be fine for 10 MX hosts that tolerate 3 connections each),
disable demand connection caching for this domain. And also:

master.cf:
slow  unix  -   -   n   -   -   smtp
  -o smtp_connection_cache_on_demand=no

main.cf:
slow_initial_destination_concurrency = 3
slow_destination_concurrency_failed_cohort_limit = 3
slow_destination_concurrency_positive_feedback = 1/5
slow_destination_concurrency_negative_feedback = 1/8

This will spread the load more evenly among the 10 (you say equal-weight)
MX hosts and will reduce the rate at which concurrency is increased
while improving error tolerance so that the destination is not throttled
prematurely.

-- 
Viktor.


Re : Re : slow transport, master.cf and maxproc value

2011-03-02 Thread myrdhin bzh
>> # in master.cf :

>> slow  unix  -   -   n   -   5   smtp

> Something like that. The "5" process limit will mean that with multiple
> "slow" domains at most 5 delivery processes will be used in total for
> all "slow" destinations. This may not be what you want. It can however
> reduce instances of a single MX host getting many connections when it
> serves multiple slow domains. The downside is a potential bottleneck.

My /etc/postfix/transport file contains one domain (domain.tld). This domain 
have 10 SMTP servers (smtp.domain.tld with round-robin DNS configuration). I 
know that each SMTP server can only accept 3 concurrency connections.

I find lot of lines like this in my mail.log :

Mar  2 20:31:04 localhost postfix/smtp[5394]: 2A5529C4429: host 
smtp.domain.tld[xxx.xxx.xxx.xxx] refused to talk to me: 421 mwinf5c20 ME Trop 
de 
connexions, veuillez verifier votre configuration. Too many connections, slow 
down. OFR004_104 [104]

How can i configure my postfix ?
thanks,
-- 
Myrdhin,






Re: Re : slow transport, master.cf and maxproc value

2011-03-02 Thread Victor Duchovni
On Wed, Mar 02, 2011 at 10:42:47PM +, myrdhin bzh wrote:

> Thanks for your response Victor.
> So, if i want max 3 connections (to domain.tld), is this configuration could 
> work (?) :
> 
> # in main.cf :
> transport_maps = hash:/etc/postfix/transport
> slow_destination_concurrency_limit=3

Yes.

> # in /etc/postfix/transport
> domain.tld slow:
> 
> # in master.cf :
> slow  unix  -   -   n   -   5   smtp

Something like that. The "5" process limit will mean that with multiple
"slow" domains at most 5 delivery processes will be used in total for
all "slow" destinations. This may not be what you want. It can however
reduce instances of a single MX host getting many connections when it
serves multiple slow domains. The downside is a potential bottleneck.

-- 
Viktor.


Re : slow transport, master.cf and maxproc value

2011-03-02 Thread myrdhin bzh
Thanks for your response Victor.
So, if i want max 3 connections (to domain.tld), is this configuration could 
work (?) :

# in main.cf :
transport_maps = hash:/etc/postfix/transport
slow_destination_concurrency_limit=3

# in /etc/postfix/transport
domain.tld slow:

# in master.cf :
slow  unix  -   -   n   -   5   smtp
thanks,
-- 
Myrdhin,






Re: slow transport, master.cf and maxproc value

2011-03-02 Thread Victor Duchovni
On Wed, Mar 02, 2011 at 11:23:45PM +0100, Myrdhin wrote:

> # in main.cf :
> transport_maps = hash:/etc/postfix/transport
>
> # in /etc/postfix/transport
> domain.tld slow:
>
> # in master.cf :
> slow  unix  -   -   n   -   5   smtp -v -o 
> smtp_destination_concurrency_limit=3 -o slow_destination_rate_delay=1

These settings are in the wrong place. These are qmgr(8) parameters,
not smtp(8) delivery agent parameters.

Furthermore, rate delay > 0 implies concurrency = 1.

> I would like to know if the maximum connections defined in the slow 
> transport are 3 or 15 (3 smtp_destination_concurrency_limit x 5 maxproc) ?

Neither, since you have not exposed the queue manager to these settings.

> If it's 15, how could i configure Postfix to make 3 max connections ?

If you want a concurrency limit > 1, you can't set a rate limit. All
concurrency control is in the queue manager so settings go in main.cf.

-- 
Viktor.