[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
On 8/27/23 20:50, Viktor Dukhovni via Postfix-users wrote: I am told that Comcast have raised the limits somewhat, and it should no longer be necessary to set the recipient limit to 1. I expect you should now be able to get away with something more reasonable, like 10 or worst-case 5, unless your server has a "known bad" reputation. I don't know whether the full RFC 5321 100 recipient limit, or the Postfix default of 50 is viable. Thank you, and thank your contact at Comcast for getting a production change rolled out so quickly! I've switched to a recipient limit of 2 for the comcast transport for now -- don't want to push my luck. Thanks again! - Bill ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: smtpd_command_filter: Bounce-never regex sample wrong?
lutz.niederer--- via Postfix-users: > Hi! > > In postconf > smtpd_command_filter section there is an example for never > bouncing mails (no DSN): > > # Bounce-never mail sink. Use notify_classes=bounce,resource,software > # to send bounced mail to the postmaster (with message body removed). > /^(RCPT\s+TO:\s*<.*>.*)\s+NOTIFY=\S+(.*)/ $1 NOTIFY=NEVER$2 > /^(RCPT\s+TO:.*)/ $1 NOTIFY=NEVER > > RFC3461 says > > notify-esmtp-value = "NEVER" / 1#notify-list-element > notify-list-element = "SUCCESS" / "FAILURE" / "DELAY" > >Notes: > >a. Multiple notify-list-elements, separated by commas, MAY appear in > a NOTIFY parameter; however, the NEVER keyword MUST appear by > itself. > > Why is there a "$2" appended in the first line of the example if only "NEVER" > is allowed? Because there may be MORE command parmeters after NOTIFY. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
On Sun, Aug 27, 2023 at 02:33:49PM -0400, Viktor Dukhovni via Postfix-users wrote: > I hope that Comcast will relax their limits to allow at least 2 (ideally > closer to 5 or 10) recipients per message so long as the sending system > does not have a "known bad" reputation. I am told that Comcast have raised the limits somewhat, and it should no longer be necessary to set the recipient limit to 1. I expect you should now be able to get away with something more reasonable, like 10 or worst-case 5, unless your server has a "known bad" reputation. I don't know whether the full RFC 5321 100 recipient limit, or the Postfix default of 50 is viable. -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: BUG: Postfix deals badly with corrected-typo in aliases :(
On Sun, Aug 27, 2023 at 04:06:18PM -0400, Viktor Dukhovni via Postfix-users wrote: > If the aliases(5) table has actually been rebuilt, and the message > is now deliverable, the background refresh is supposed to happen: > > address_verify_negative_refresh_time (default: 3h) >The time after which a failed address verification probe needs to be >refreshed. > >Specify a non-zero time value (an integral value plus an optional >one-letter suffix that specifies the time unit). Time units: s >(seconds), m (minutes), h (hours), d (days), w (weeks). The default >time unit is h (hours). > >This feature is available in Postfix 2.1 and later. > > > https://github.com/vdukhovni/postfix/blob/v3.8.1/postfix/src/verify/verify.c#L512-L550 > > If you have detailed log evidence of refresh probes not happening, > please share. FWIW, I am running an experiment on my end, asking the verify(8) service to verify a non-existent virtual mailbox: #define DEL_RCPT_STAT_OK0 #define DEL_RCPT_STAT_DEFER 1 #define DEL_RCPT_STAT_BOUNCE2 #define DEL_RCPT_STAT_TODO 3 # date; postconf address_verify_negative_refresh_time; printf 'request\0query\0address\0postfix-lus...@dukhovni.org\0\0' | nc -NU /var/spool/postfix/private/verify | tr '\0' '\n' Sun Aug 27 16:35:39 EDT 2023 address_verify_negative_refresh_time = 1200s protocol address_verification_prrotocol status 0 recipient_status 3 reason Address verification in progress The initial query returns a pending (TODO) result, which in short order resolves to an error (BOUNCE) status: Aug 27 16:35:39 amnesiac postfix/cleanup[55691]: C8FA712A3DC: message-id=<...> Aug 27 16:35:39 amnesiac postfix/qmgr[55681]: C8FA712A3DC: from=, size=273, nrcpt=1 (queue active) Aug 27 16:35:39 amnesiac postfix/error[55692]: C8FA712A3DC: to=, relay=none, delay=0.02, delays=0/0.02/0/0, dsn=5.1.1, status=undeliverable (User unknown in virtual alias table) Aug 27 16:35:39 amnesiac postfix/qmgr[55681]: C8FA712A3DC: removed # date; postconf address_verify_negative_refresh_time; printf 'request\0query\0address\0postfix-lus...@dukhovni.org\0\0' | nc -NU /var/spool/postfix/private/verify | tr '\0' '\n' Sun Aug 27 16:36:18 EDT 2023 address_verify_negative_refresh_time = 1200s protocol address_verification_prrotocol status 0 recipient_status 2 reason User unknown in virtual alias table After waiting out the 20 minute refresh TTL, another verify(8) query returns the same result: # date; postconf address_verify_negative_refresh_time; printf 'request\0query\0address\0postfix-lus...@dukhovni.org\0\0' | nc -NU /var/spool/postfix/private/verify | tr '\0' '\n' Sun Aug 27 16:55:44 EDT 2023 address_verify_negative_refresh_time = 1200s protocol address_verification_prrotocol status 0 recipient_status 2 reason User unknown in virtual alias table However, there's also a background probe: Aug 27 16:55:44 amnesiac postfix/cleanup[55843]: 9E78612A27D: message-id=<...> Aug 27 16:55:44 amnesiac postfix/qmgr[55681]: 9E78612A27D: from=, size=273, nrcpt=1 (queue active) Aug 27 16:55:44 amnesiac postfix/error[55844]: 9E78612A27D: to=, relay=none, delay=0.02, delays=0/0.02/0/0, dsn=5.1.1, status=undeliverable (User unknown in virtual alias table) Aug 27 16:55:44 amnesiac postfix/qmgr[55681]: 9E78612A27D: removed If I had already added the problem user, this probe would have updated the database to indicate that the user exists. After dropping the refresh time to 120s, and adding the table entry, we see: # date; postconf address_verify_negative_refresh_time; printf 'request\0query\0address\0postfix-lus...@dukhovni.org\0\0' | nc -NU /var/spool/postfix/private/verify | tr '\0' '\n' Sun Aug 27 17:02:28 EDT 2023 address_verify_negative_refresh_time = 120s protocol address_verification_prrotocol status 0 recipient_status 2 reason User unknown in virtual alias table But now the resulting background probe succeeds: Aug 27 17:02:28 amnesiac postfix/cleanup[55936]: 29A8112A603: message-id=<...> Aug 27 17:02:28 amnesiac postfix/qmgr[55921]: 29A8112A603: from=, size=273, nrcpt=1 (queue active) Aug 27 17:02:28 amnesiac postfix/virtual[55937]: 29A8112A603: to=<...>, orig_to=, relay=virtual, delay=0.03, delays=0/0.02/0/0, dsn=2.0.0, status=deliverable (delivers to maildir) Aug 27 17:02:28 amnesiac postfix/qmgr[55921]: 29A8112A603: removed So that, shortly after verification returns sucess (OK): # date; postconf address_verify_negative_refresh_time; printf 'request\0query\0address\0postfix-lus...@dukhovni.org\0\0' | nc -NU /var/spool/postfix/private/verify | tr '\0' '\n' Sun Aug 27 17:06:08 EDT 2023 address_verify_negative_refresh_time = 120s protocol
[pfx] Re: BUG: Postfix deals badly with corrected-typo in aliases :(
On Sun, Aug 27, 2023 at 01:41:19PM -0600, Pete Holzmann wrote: > Ummm... Viktor, how many people do *you* think have read the fine > documentation on every verification option they use in their main.cf > restriction configurations? I don't know. What I do know is that using features whose documentation has not been read is liable to lead to naïvely surprising behaviour. :-( > Beyond that, from experience this statement is false: > > >As documented, but there should also be a background refresh every 3 > >hours, so that the second try 3h after correcting the problem, will > >succeed: > > >address_verify_negative_expire_time = 3d > >address_verify_negative_refresh_time = 3h > >address_verify_positive_expire_time = 31d > >address_verify_positive_refresh_time = 7d If the aliases(5) table has actually been rebuilt, and the message is now deliverable, the background refresh is supposed to happen: address_verify_negative_refresh_time (default: 3h) The time after which a failed address verification probe needs to be refreshed. Specify a non-zero time value (an integral value plus an optional one-letter suffix that specifies the time unit). Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks). The default time unit is h (hours). This feature is available in Postfix 2.1 and later. https://github.com/vdukhovni/postfix/blob/v3.8.1/postfix/src/verify/verify.c#L512-L550 If you have detailed log evidence of refresh probes not happening, please share. > I have 100% default values. Ten hours after correcting the problem, > it still failed. Perhaps a bug? During those 10 hours, how many deliveries were attempted to the problem address after the source table was updated and the indexed "database" file rebuilt. The "refresh" probes only run in response to delivery attempts, in the background, after returning a cached response to the SMTP client. > >Only if you haven't read the documentation. > > Whether or not the fine documentation has been read, no information > in the log, even at log-level 10, points to the verification cache. Well, "log level 10" (whatever that means) is a good way of not seeing the forest for the trees. At most one "-v" is generally all that one might temporarily enable to get more insight into the details of message processing. In this case for the smtpd(8) and/or verify(8) services. > >No, newaliases have nothing to do with this. This is entirely material > >for ADDRESS_VERIFICATION_README and verify(8), but perhaps the timer > >parameters could be also mentioned prominently in the README file. > > Why would it not make sense for newaliases to trigger a refresh? > Clearly, newaliases implies that aliases have been changed, so it is > quite possible that prior failures are now successes. ;) Because there are over 1000 tunable parameters, with many of them potentially affecting the deliverability of messages to a recipient. There nothing particularly remarkable about the content of aliases(5). The behaviour of recipient verification is documented in verify(8) and ADDRESS_VERIFICATION_README, with per-parameter details in postconf(5). -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
On Sun, Aug 27, 2023 at 11:12:03AM -0700, Bill Sommerfeld via Postfix-users wrote: > On 8/27/23 00:13, Wietse Venema via Postfix-users wrote: > > Would it be sufficient to never send more than 1 recipient per > > mesage, thus never trigger their temporary "block all mail" strategy, > > and avoid the need for the kludges described here? > > I set smtpserial_destination_rate_delay=60s to put a 60 message/hour > ceiling on the instantaneous rate, but the documentation on that tunable > says: > > With a corresponding per-destination recipient limit equal to 1, > the rate delay specifies the time between deliveries to the same > recipient. Different recipients are delivered in parallel, subject > to the process limits specified in master.cf. > > which suggests that even with a 1-process limit for the slow mailer it > might still deliver to different recipients back-to-back and only wait > the full time between two deliveries to the same recipient. That's exactly what it means. > In other words, it is unclear from the documentation whether the > destination_rate_delay is implemented in the transport (consuming a > transport process while it waits) or in the queue manager. All rate limits are in the queue manager, the process limit is just that. It couldn't really be otherwise, and it makes no sense to hog an idle process. Instead, the queue manager doesn't resume delivery to a logical destination (nexthop or user@nexthop when the recipient limit is equal to 1) until the rate delay timer expires. Meanwhile other deliveries for the same transport may proceed if not also blocked by either concurrency or rate limits. I hope that Comcast will relax their limits to allow at least 2 (ideally closer to 5 or 10) recipients per message so long as the sending system does not have a "known bad" reputation. -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
Thanks for your prompt responses. On 8/27/23 00:13, Wietse Venema via Postfix-users wrote: Would it be sufficient to never send more than 1 recipient per mesage, thus never trigger their temporary "block all mail" strategy, and avoid the need for the kludges described here? That's unclear. I'm trying to both avoid the quirks of Comcast's observed behavior and stay well away from their documented limits at http://postmaster.comcast.net/smtp-error-codes.php#RL01 where their lowest-tier rate limit is 120 messages/hour. I set smtpserial_destination_rate_delay=60s to put a 60 message/hour ceiling on the instantaneous rate, but the documentation on that tunable says: With a corresponding per-destination recipient limit equal to 1, the rate delay specifies the time between deliveries to the same recipient. Different recipients are delivered in parallel, subject to the process limits specified in master.cf. which suggests that even with a 1-process limit for the slow mailer it might still deliver to different recipients back-to-back and only wait the full time between two deliveries to the same recipient. In other words, it is unclear from the documentation whether the destination_rate_delay is implemented in the transport (consuming a transport process while it waits) or in the queue manager. On 8/27/23 00:58, Wietse Venema via Postfix-users wrote: > I think that the 'slow' example should handle this: Indeed, as that is close to what I ended up with. Thanks for the pointer -- I hadn't noticed the "slow" example in QSHAPE_README.html as the file seemed geared only to large-volume sites which I emphatically am not. I'd be happy to write up a section on coping with rate limiting for inclusion in SOHO_README if it would help others. - Bill ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: smtpd_command_filter: Bounce-never regex sample wrong?
On Sun, Aug 27, 2023 at 10:25:10AM +0200, lutz.niederer--- via Postfix-users wrote: > In postconf > smtpd_command_filter section there is an example for never > bouncing mails (no DSN): > > # Bounce-never mail sink. Use notify_classes=bounce,resource,software > # to send bounced mail to the postmaster (with message body removed). > /^(RCPT\s+TO:\s*<.*>.*)\s+NOTIFY=\S+(.*)/ $1 NOTIFY=NEVER$2 > /^(RCPT\s+TO:.*)/ $1 NOTIFY=NEVER > > RFC3461 says > > notify-esmtp-value = "NEVER" / 1#notify-list-element > notify-list-element = "SUCCESS" / "FAILURE" / "DELAY" > >Notes: > >a. Multiple notify-list-elements, separated by commas, MAY appear in > a NOTIFY parameter; however, the NEVER keyword MUST appear by > itself. > > Why is there a "$2" appended in the first line of the example if only "NEVER" > is allowed? Consider: RCPT TO: NOTIFY=SUCCESS SIZE=12345 taking into account that "\S+" greedily matches non-whitespace. [ The "$2" suffix will necessarily be empty or will start with whitespace. ] -- Viktor. ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
On Sun, Aug 27, 2023 at 03:13:43AM -0400, Wietse Venema via Postfix-users wrote:
> Bill Sommerfeld via Postfix-users:
> > About three years ago there was a thread on postfix-users ("Comcast 421
> > throttling multiple recipients") discussing a low-traffic site having
> > difficulties sending to multiple recipients at comcast in a single smtp
> > session. The thread starts here:
> >
> > https://www.mail-archive.com/postfix-users@postfix.org/msg88394.html
> >
> > and it appears to have died out without consensus on what was going on.
> >
> > I believe I understand what the original poster was seeing because it
> > just happened to me. Having a way to disable the special per-recipient
> > behavior when ${transport}_destination_recipient_limit=1 would be very
> > useful in working around quirky receiver like this.
>
> Would it be sufficient to never send more than 1 recipient per
> mesage, thus never trigger their temporary "block all mail" strategy,
> and avoid the need for the kludges described here?
Meanwhile, I pinged to a Comcast contact. He mentioned that they're
reachable via the "mailop" list, and may perhaps consider loosening the
limits a bit. I hope that'll happen, and small Postfix systems with
neutral/unknown "reputation" wouldn't have to jump through excessive
hoops just to deliver an occasional multi-recipient message.
--
Viktor.
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] smtpd_command_filter: Bounce-never regex sample wrong?
Hi! In postconf > smtpd_command_filter section there is an example for never bouncing mails (no DSN): # Bounce-never mail sink. Use notify_classes=bounce,resource,software # to send bounced mail to the postmaster (with message body removed). /^(RCPT\s+TO:\s*<.*>.*)\s+NOTIFY=\S+(.*)/ $1 NOTIFY=NEVER$2 /^(RCPT\s+TO:.*)/ $1 NOTIFY=NEVER RFC3461 says notify-esmtp-value = "NEVER" / 1#notify-list-element notify-list-element = "SUCCESS" / "FAILURE" / "DELAY" Notes: a. Multiple notify-list-elements, separated by commas, MAY appear in a NOTIFY parameter; however, the NEVER keyword MUST appear by itself. Why is there a "$2" appended in the first line of the example if only "NEVER" is allowed? Thanks! -lutzn ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
Wietse Venema via Postfix-users:
> Bill Sommerfeld via Postfix-users:
> > About three years ago there was a thread on postfix-users ("Comcast 421
> > throttling multiple recipients") discussing a low-traffic site having
> > difficulties sending to multiple recipients at comcast in a single smtp
> > session. The thread starts here:
> >
> > https://www.mail-archive.com/postfix-users@postfix.org/msg88394.html
> >
> > and it appears to have died out without consensus on what was going on.
> >
> > I believe I understand what the original poster was seeing because it
> > just happened to me. Having a way to disable the special per-recipient
> > behavior when ${transport}_destination_recipient_limit=1 would be very
> > useful in working around quirky receiver like this.
>
> Would it be sufficient to never send more than 1 recipient per
> mesage, thus never trigger their temporary "block all mail" strategy,
> and avoid the need for the kludges described here?
I think that the 'slow' example should handle this:
/etc/postfix/master.cf:
# Don't immediately try an alternate MX after 4xx server response.
slow unix - - - - 1 smtp
-o { smtp_mx_session_limit = 1 }
# Execute "postmap hash:/etc/postfix/transport" after editing the file,
# then "postfix reload".
/etc/postfix/transport:
comcast.net slow:
/etc/postfix/main.cf:
transport_maps = hash:/etc/postfix/transport
# Don't send multi-recipient messages over the 'slow' transport.
slow_destination_recipient_limit = 1
# Don't send parallel messages over the 'slow' transport.
slow_destination_concurrency_limit = 1
# Don't send messages back to back over the 'slow' transport.
slow_destination_rate_delay = 1
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Comcast still 421 throttling (RL000001) multiple recipients.
Bill Sommerfeld via Postfix-users:
> About three years ago there was a thread on postfix-users ("Comcast 421
> throttling multiple recipients") discussing a low-traffic site having
> difficulties sending to multiple recipients at comcast in a single smtp
> session. The thread starts here:
>
> https://www.mail-archive.com/postfix-users@postfix.org/msg88394.html
>
> and it appears to have died out without consensus on what was going on.
>
> I believe I understand what the original poster was seeing because it
> just happened to me. Having a way to disable the special per-recipient
> behavior when ${transport}_destination_recipient_limit=1 would be very
> useful in working around quirky receiver like this.
Would it be sufficient to never send more than 1 recipient per
mesage, thus never trigger their temporary "block all mail" strategy,
and avoid the need for the kludges described here?
Wietse
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
