RE: missing log entries for old version of postfix

[Sean Hennessey]

Wietse,

Great call:
Feb  9 12:58:01 ip-10-104-84-189 journal: Suppressed 5141 messages from 
/user.slice/user-0.slice
Feb  9 13:00:00 ip-10-104-84-189 journal: Suppressed 7542 messages from 
/user.slice/user-0.slice
Feb  9 13:00:37 ip-10-104-84-189 journal: Suppressed 22 messages from 
/user.slice/user-0.slice
Feb  9 13:02:18 ip-10-104-84-189 journal: Suppressed 3853 messages from 
/user.slice/user-0.slice

I need to go look at my new postfix machines to make sure those aren't 
happening there.

Those machines have mail_version = 3.6.4

Time to look in the docs to see if that's turned on.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Wietse Venema
Sent: Friday, February 10, 2023 12:34 PM
To: Postfix users 
Subject: Re: missing log entries for old version of postfix

Sean Hennessey:
> I have a machine running a very old version of postfix. In trying to 
> troubleshoot a discrepancy between what some reports I was pulling 
> from the postfix logs vs what my upstream relay was saying I was 
> sending I noticed something that looks off to me.
 
Logging from qmgr and 

> /var/log/maillog:Feb  9 13:00:03 ip-10-104-84-189 postfix/smtp[3788]: 
> 690834E92AC: to=us...@gmail.com<mailto:us...@gmail.com>, 
> relay=send.hidden.com[x.x.x.x]:587, delay=5.6, 
> delays=0.01/5.1/0.29/0.19, dsn=2.0.0, status=sent (250 OK ) 
> /var/log/maillog:Feb  9 13:00:03 ip-10-104-84-189 postfix/qmgr[21067]: 
> 690834E92AC: removed

Similar logging omitted because it has an identical pattern.

All three messages were queued at the same time, 12:59:58, and were delivered 
between 13:00:03 and 13:00:06.  

All three messages would have produced cleanup logging and qmgr logging at the 
time that the messages were queued.

These are missing:

postfix/cleanup[pid]: queueid: message-id=
postfix/qmgr[pid]: queueid: from=, size=nn, nrcpt=nn (queue active)

Most plausible explanations:

- The logfile was rotated between 12:59:58 and 13:00:03.

- Your systemd is throttling Postfix logs, and threw away the logging
  that was generated around 12:59:58. This is one reason why I
  implemented direct-to-file logging in Postfix 3.4.

Wietse

missing log entries for old version of postfix

[Sean Hennessey]

I have a machine running a very old version of postfix. In trying to 
troubleshoot a discrepancy between what some reports I was pulling from the 
postfix logs vs what my upstream relay was saying I was sending I noticed 
something that looks off to me.

[ssm-user@ip-10-104-84-189 ~]$ sudo grep -E 
"C78B04FBDFB|7AECF4E97A8|690834E92AC" /var/log/maillog*
/var/log/maillog:Feb  9 13:00:03 ip-10-104-84-189 postfix/smtp[3788]: 
690834E92AC: to=us...@gmail.com, 
relay=send.hidden.com[x.x.x.x]:587, delay=5.6, delays=0.01/5.1/0.29/0.19, 
dsn=2.0.0, status=sent (250 OK )
/var/log/maillog:Feb  9 13:00:03 ip-10-104-84-189 postfix/qmgr[21067]: 
690834E92AC: removed
/var/log/maillog:Feb  9 13:00:03 ip-10-104-84-189 postfix/smtp[3800]: 
7AECF4E97A8: to=us...@gmail.com, 
relay=send.hidden.com[x.x.x.x]:587, delay=5.9, delays=0/5.4/0.32/0.24, 
dsn=2.0.0, status=sent (250 OK )
/var/log/maillog:Feb  9 13:00:03 ip-10-104-84-189 postfix/qmgr[21067]: 
7AECF4E97A8: removed
/var/log/maillog:Feb  9 13:00:05 ip-10-104-84-189 postfix/smtp[3798]: 
C78B04FBDFB: to=us...@gmail.com, 
relay=send.hidden.com[x.x.x.x]:587, delay=7.3, delays=0.01/6.9/0.29/0.12, 
dsn=2.0.0, status=sent (250 OK )
/var/log/maillog:Feb  9 13:00:05 ip-10-104-84-189 postfix/qmgr[21067]: 
C78B04FBDFB: removed

That's the only records in the logs for those queue id's. There isn't any of 
the normal inbound stuff I'm used to seeing. I've shifted sending to another 
box running a newer version of postfix for now, but I'm curious if there is a 
known bug or if I've got something set up weird. It's not happening all the 
time, but at various times.

[ssm-user@ip-10-104-84-189 ~]$ postconf mail_version
mail_version = 2.10.1

Re: append_dot_mydomain, how to make it work

[Sean Hennessey]

Viktor / Wietse,

Thanks a million for this. It seems to work great in my test lab. I'm going to 
move it forward on a secondary server before going to prod w/ it.

Sean

From: owner-postfix-us...@postfix.org  on 
behalf of Viktor Dukhovni 
Sent: Sunday, January 29, 2023 6:31 PM
To: postfix-users@postfix.org 
Subject: Re: append_dot_mydomain, how to make it work

On Sun, Jan 29, 2023 at 05:57:45PM -0500, Wietse Venema wrote:

> > > Is there a way to accomplish that?
> >
> > Yes:
> >
> >   master.cf:
> >
> > smtp   inet  n   -   n   -   -   smtpd
> > -o { append_at_myorigin = yes }
> > -o { rewrite_service_name = bh-rewrite }
> > -o { cleanup_service_name = bh-rewrite }
>
> (bh-cleanup)

Indeed.  Apologies to the OP about the typo.

> > bh-cleanup unix  n   -   n   -   0   cleanup
> > -o { append_at_myorigin = yes }
> > -o { rewrite_service_name = bh-rewrite }
> > ...
> > rewriteunix  -   -   n   -   -   trivial-rewrite
> > bh-rewrite unix  -   -   n   -   -   trivial-rewrite
> > -o { myorigin = blackhole.invalid }
> >
> > This will affect all unqualified addresses that come in via the SMTP
> > service in question.  All other sources remain unchanged.
> >
> > You could even use some sort of NAT rules to direct incoming
> > SMTP traffic from just the problem clients to the custom SMTP
> > service on an alternate IP:port.
>
> I suspect that "-o { append_at_myorigin = yes }" is needed only for
> the bh-rewrite service. The append_at_myorigin value is is referenced
> only in the trivial-rewrite daemon, and is not sent via IPC.

Yes, and I even checked where the parameter is used, but then wrote my
post apparently on auto-pilot.  So the corrected "recipe" (modulo any
new typos, testing recommended) should be:

smtp   inet  n   -   n   -   -   smtpd
-o { rewrite_service_name = bh-rewrite }
-o { cleanup_service_name = bh-cleanup }
...
cleanupunix  n   -   n   -   0   cleanup
bh-cleanup unix  n   -   n   -   0   cleanup
-o { rewrite_service_name = bh-rewrite }
...
rewriteunix  -   -   n   -   -   trivial-rewrite
bh-rewrite unix  -   -   n   -   -   trivial-rewrite
-o { append_at_myorigin = yes }
-o { myorigin = blackhole.invalid }

--
Viktor.

Re: append_dot_mydomain, how to make it work

[Sean Hennessey]

Viktor,

What I'm looking for is the envelope recipient. I need a way to force an 
unqualified to address to a domain I can blackhole. I've got an application 
that feeds into these systems that will allow its users to enter badly formed 
email addresses. What I want to do is to just swallow those in postfix. What I 
was attempting to do, was get the envelope to to be forced to @blackhole.local, 
and then I just discard that w/ a transport map.

Is there a way to accomplish that?



From: owner-postfix-us...@postfix.org  on 
behalf of Viktor Dukhovni 
Sent: Sunday, January 29, 2023 2:28 PM
To: postfix-users@postfix.org 
Subject: Re: append_dot_mydomain, how to make it work

On Sun, Jan 29, 2023 at 06:32:12PM +0000, Sean Hennessey wrote:

> That was one of my test cases. I just tried it again;
>
> $ postconf |
>   grep -E 
> "append_at_my|remote_header_rewrite|local_header_rewrite|inet_interfaces"
> append_at_myorigin = yes
> inet_interfaces = all
> local_header_rewrite_clients =
> remote_header_rewrite_domain = blackhole.local

Note the string "_header_" in the last two parameters.

> 220 postfix-warming-c.us-east-2.compute.internal ESMTP Postfix (Ubuntu)
> helo me
> 250 postfix-warming-c.us-east-2.compute.internal
> mail from:
> 250 2.1.0 Ok
> rcpt to:
> 250 2.1.5 Ok
> data
> 354 End data with .
> .
> 250 2.0.0 Ok: queued as ADA723FAB4

You're submitting an empty message (no headers), with an unqualified
*envelope* recipient address.  And you log samples in any case only
provide evidence of the resulting envelope.

> Jan 29 18:24:29 postfix-warming-c postfix/local[16927]: ADA723FAB4:
>   to=,
>   orig_to=,
>   relay=local, delay=6.8, delays=6.8/0/0/0.01,
>   dsn=5.1.1, status=bounced (unknown user: "x")

The envelope recipient was qualified with @$myorigin as configured.  If
you were expecting something else, you've misinterpreted the
documentation.

--
Viktor.

Re: append_dot_mydomain, how to make it work

[Sean Hennessey]

Wietse,

That was one of my test cases. I just tried it again;

ssm-user@postfix-warming-c:~$ postconf | grep -E 
"append_at_my|remote_header_rewrite|local_header_rewrite|inet_interfaces"
append_at_myorigin = yes
inet_interfaces = all
local_header_rewrite_clients =
remote_header_rewrite_domain = blackhole.local

220 postfix-warming-c.us-east-2.compute.internal ESMTP Postfix (Ubuntu)
helo me
250 postfix-warming-c.us-east-2.compute.internal
mail from:
250 2.1.0 Ok
rcpt to:
250 2.1.5 Ok
data
354 End data with .
.
250 2.0.0 Ok: queued as ADA723FAB4

sudo grep ADA723FAB4 /var/log/mail.log
Jan 29 18:24:27 postfix-warming-c postfix/smtpd[16934]: input attribute value: 
ADA723FAB4
Jan 29 18:24:27 postfix-warming-c postfix/smtpd[16934]: ADA723FAB4: 
client=ip-10-104-87-249.us-east-2.compute.internal[10.104.87.249]
Jan 29 18:24:29 postfix-warming-c postfix/cleanup[16925]: ADA723FAB4: 
message-id=<20230129182427.ada723f...@postfix-warming-c.us-east-2.compute.internal>
Jan 29 18:24:29 postfix-warming-c postfix/qmgr[16918]: ADA723FAB4: 
from=, size=383, nrcpt=1 (queue active)
Jan 29 18:24:29 postfix-warming-c postfix/smtpd[16934]: > 
ip-10-104-87-249.us-east-2.compute.internal[10.104.87.249]: 250 2.0.0 Ok: 
queued as ADA723FAB4
Jan 29 18:24:29 postfix-warming-c postfix/local[16927]: ADA723FAB4: 
to=, orig_to=, 
relay=local, delay=6.8, delays=6.8/0/0/0.01, dsn=5.1.1, status=bounced (unknown 
user: "x")
Jan 29 18:24:29 postfix-warming-c postfix/bounce[16936]: ADA723FAB4: sender 
non-delivery notification: 9D6723FABC
Jan 29 18:24:29 postfix-warming-c postfix/qmgr[16918]: ADA723FAB4: removed



From: owner-postfix-us...@postfix.org  on 
behalf of Wietse Venema 
Sent: Sunday, January 29, 2023 9:03 AM
To: postfix-users@postfix.org 
Subject: Re: append_dot_mydomain, how to make it work

> What I'm looking for is a way to force a rewrite of ADDRESSES THAT
> DON'T HAVE AN @DOMAIN that are coming into this machine from other
> computers.

Use append_at_myorigin:

append_at_myorigin (for addresses without *domain*),

Not append_dot_mydomain:

append_dot_mydomain (for addresses without *parent domain*)

Specify the forced domain with:

remote_header_rewrite_domain (default: empty)

And make sure that it happens only for clients that don't match:

local_header_rewrite_clients (which depends on SMTP client IP
address, 10.104.87.249 in your test).

Wietse

append_dot_mydomain, how to make it work

[Sean Hennessey]

I've tried just about every combination of these configs from the docs:
append_dot_mydomain (default: Postfix ≥ 3.0: no, Postfix < 3.0: yes)

With locally submitted mail, append the string 
".$mydomain" to addresses 
that have no ".domain" information. With remotely submitted mail, append the 
string 
".$remote_header_rewrite_domain"
 instead.

Note 1: this feature is enabled by default. If disabled, users will not be able 
to send mail to "user@partialdomainname" but will have to specify full domain 
names instead.

Note 2: with Postfix version 2.2, message header address rewriting happens only 
when one of the following conditions is true:

  *   The message is received with the Postfix 
sendmail(1) command,
  *   The message is received from a network client that matches 
$local_header_rewrite_clients,
  *   The message is received from the network, and the 
remote_header_rewrite_domain
 parameter specifies a non-empty value.

To get the behavior before Postfix version 2.2, specify 
"local_header_rewrite_clients
 = static:all".

but none of them are seeming to work.

ii  postfix 3.6.4-1ubuntu1  
arm64High-performance mail transport agent
-
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = yes
biff = no
compatibility_level = 3.6
debug_peer_list = 10.104.87.249
html_directory = /usr/share/doc/postfix/html
inet_interfaces = all
inet_protocols = all
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
mydestination = $myhostname, postfix-warming-c, localhost.localdomain, , 
localhost
myhostname = postfix-warming-c.us-east-2.compute.internal
mynetworks = 127.0.0.0/8, 10.20.0.0/22, 10.20.4.0/22, 10.20.8.0/22, 
10.20.32.0/19, 10.20.64.0/19, 10.20.96.0/19, 10.20.128.0/22, 10.20.132.0/22, 
10.20.136.0/22, 10.104.80.0/24, 10.104.81.0/24, 10.104.82.0/24, 10.104.84.0/23, 
10.104.86.0/23, 10.104.88.0/23, 10.104.90.0/24, 10.104.91.0/24, 10.104.92.0/24, 
10.108.0.0/22, 10.108.4.0/22, 10.108.8.0/22, 10.108.32.0/19, 10.108.64.0/19, 
10.108.96.0/19, 10.108.128.0/22, 10.108.132.0/22, 10.108.136.0/22, 
10.159.0.0/27, 10.159.0.32/27, 10.159.0.64/27, 10.159.0.128/25, 10.159.1.0/25, 
10.159.1.128/25, 10.159.2.0/25, 10.159.2.128/25, 10.159.3.0/25, 
10.159.3.128/27, 10.159.3.160/27, 10.159.3.192/27
readme_directory = /usr/share/doc/postfix
recipient_delimiter = +
relayhost =
remote_header_rewrite_domain = blackhole.local
smtp_tls_CApath = /etc/ssl/certs
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated 
defer_unauth_destination
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_security_level = may
transport_maps = hash:/etc/postfix/transport
-

helo me
250 postfix-warming-c.us-east-2.compute.internal
mail from:
250 2.1.0 Ok
rcpt to:
550 5.1.1 : Recipient address rejected: User unknown in local recipient 
table
421 4.4.2 postfix-warming-c.us-east-2.compute.internal Error: timeout exceeded
Connection closed by foreign host.
-

If I turn on local_recipient_maps =
then I don't get the 550, but then postfix tries to deliver mail to 
blah@$myhostname.

220 postfix-warming-c.us-east-2.compute.internal ESMTP Postfix (Ubuntu)
helo me
250 postfix-warming-c.us-east-2.compute.internal
mail from:
250 2.1.0 Ok
rcpt to:
250 2.1.5 Ok
data
354 End data with .
.
250 2.0.0 Ok: queued as 2AF003FA91


Jan 29 07:10:51 postfix-warming-c postfix/smtpd[15796]: input attribute value: 
2AF003FA91
Jan 29 07:10:51 postfix-warming-c postfix/smtpd[15796]: 2AF003FA91: 
client=ip-10-104-87-249.us-east-2.compute.internal[10.104.87.249]
Jan 29 07:10:53 postfix-warming-c postfix/cleanup[15780]: 2AF003FA91: 
message-id=<20230129071051.2af003f...@postfix-warming-c.us-east-2.compute.internal>
Jan 29 07:10:53 postfix-warming-c postfix/qmgr[15772]: 2AF003FA91: 
from=, size=386, nrcpt=1 (queue active)
Jan 29 07:10:53 postfix-warming-c postfix/smtpd[15796]: > 
ip-10-104-87-249.us-east-2.compute.internal[10.104.87.249]: 250 2.0.0 Ok: 
queued as 2AF003FA91
Jan 29 07:10:53 postfix-warming-c postfix/local[15782]: 2AF003FA91: 
to=, orig_to=, 
relay=local, delay=10, delays=10/0/0/0, dsn=5.1.1, status=bounced (unknown 
user: "blah")
Jan 29 07:10:53 postfix-warming-c postfix/bounce[15784]: 2AF003FA91: sender 
non-delivery notification: 6533F3FAB5
Jan 29 07:10:53 postfix-warming-c postfix/qmgr[15772]: 2AF003FA9

mail queued by transport

[Sean Hennessey]

Is there a way to see the transport that queued mail is using?

I just created a new transport and pointed a domain to it. The server also had 
mail for that domain queued up under the old transport.

Is there any way I can find out what transport the mail is using?

I didn't see transport in the field list when I ran a json dump of the queue.

Re: Understanding concurrency limits

[Sean Hennessey]

Thanks,I did realize after I sent the email that what was probably happening 
was the delay was the overiding controller, and not working as in addition as I 
thought it would.

Is there a way to do combine the delay w/ the concurrency?

Like two simultaneous connections each doing one email per rate delay?

I'm trying to help some folks out w/ sending to yahoo and trying to see what 
options I have as far as rate control.

From: owner-postfix-us...@postfix.org  on 
behalf of Wietse Venema 
Sent: Tuesday, January 17, 2023 7:17 PM
To: Postfix users 
Subject: Re: Understanding concurrency limits

Sean Hennessey:
> In master.cf
> smtp-tar unix  --   y   -   1   smtp
> -o syslog_name=postfix/$service_name
>
> In main.cf
> smtp-tar_destination_rate_delay = 600s

RTFM, this puts 600s delay between deliveries as in:

deliver one meessage
wait 600s
deliver one meessage
etc.

The scheuler could warn when concurrency >1 will be ignored, but
given that "default_destination_concurrency_limit = 20", that would
create a lot of noise.

Wietse

Understanding concurrency limits

[Sean Hennessey]

In master.cf
smtp-tar unix  --   y   -   1   smtp
-o syslog_name=postfix/$service_name

In main.cf
smtp-tar_destination_rate_delay = 600s
smtp-tar_destination_concurrency_limit = 3
smtp-tar_destination_recipient_limit = 2
smtp-tar_initial_destination_concurrency=2

In transports
yahoo.com   smtp-tar:

My understanding is that w/ the above settings, the first sends going to 
@yahoo.com should fire off two simultaneous smtp connections. Then at some 
point, it'll build up to three. I don't see that.

I reload postfix at 21:48 and this is what I see in the logs for outbound 
yahoo.com

Jan 17 21:48:01 postfix2 postfix/smtp-tar/smtp[2541820]: B142044CB2: 
to=, relay=mta5.am0.yahoodns.net[98.136.96.74]:25, delay=4255, 
delays=4255/0.06/0.15/0.4, dsn=2.0.0, status=sent (250 ok dirdel)
Jan 17 21:58:06 postfix2 postfix/smtp-tar/smtp[2542451]: EC44B44751: 
to=, relay=mta6.am0.yahoodns.net[67.195.204.79]:25, 
delay=17699, delays=17093/601/0.23/5.1, dsn=2.0.0, status=sent (250 ok dirdel)
Jan 17 22:08:07 postfix2 postfix/smtp-tar/smtp[2543075]: 1135144D94: 
to=, relay=mta6.am0.yahoodns.net[98.136.96.91]:25, 
delay=2122, delays=914/1206/0.16/0.67, dsn=2.0.0, status=sent (250 ok dirdel)

I'm only seeing one outbound at a time, every ten minutes.

Am I doing something wrong here?

Re: make transport decisions based on headers not envelope

[Sean Hennessey]

I was using the sender_dependent_default_transport_maps to pick off what I 
thought was going to be the interesting from domain. The good news is that this 
mail is coming from customer applications. It's not coming from regular user 
mail clients. So I can guarantee there is going to be a single from header. The 
recipients are also never local. This machine is a pure relay.

Looks like it's time to learn me some milter. Can you recommend any good places 
to start on walking me through building one. I've got a programming background, 
so coding doesn't scare me.

Thanks again.


From: owner-postfix-us...@postfix.org  on 
behalf of Viktor Dukhovni 
Sent: Thursday, January 12, 2023 11:48 PM
To: postfix-users@postfix.org 
Subject: Re: make transport decisions based on headers not envelope

On Fri, Jan 13, 2023 at 03:22:41AM +, Sean Hennessey wrote:

> I'm back to the experts for some question. I'm trying to help dig some
> folks out of hole. Is there anyway possible to make a decision on
> where to send mail based on a friendly from header instead of the
> envelope from?

NO! That way lies madness and endless mail loops.  Or, more precisely,
absolutely not when deciding which way to route the *recipients* of
the message.

> It turns out that their system using an envelope from that's the same
> for everyone, and just uses the customer specific from in the From:
> header. That's probably the better thing to be doing, but it just
> burnt me because the system I'm most recently working on matches
> aligns the envelope from and the header from.

Presumably, this is a case of trying to use:

sender_dependendent_default_transport_maps

this could in principle be based on the "From:" header without risking
loops, however there's no such feature in Postfix, and there are various
potential complications:

- What happens when there's both a "Sender:" and a "From:" header?
- What happens when "Resent-From:" is present?
- What happens in the unusual case where there are multiple "From:"
  addresses (allowed by RFC822, with "Sender:" required).
- What happens when there is no "From:" header?  (And Postfix adds
  "Apparently-From: ").
- ...

So, provided the accepted recipients are *NEVER EVER* local or any other
address class, you'd want to use a milter to prepend a header that
indicates the exit transport, and then use milter_header_checks to
specify a "FILTER" action that forces all recipients to that transport.

This is a bit ugly, but puts the ball in your court (all the messy
details are in your milter, whatever you decide goes).  Don't forget,
the precondition, there must never be any recipients in any address
class other than "default" or else routing will be wrong.

--
Viktor.

make transport decisions based on headers not envelope

[Sean Hennessey]

I'm back to the experts for some question. I'm trying to help dig some folks 
out of hole. Is there anyway possible to make a decision on where to send mail 
based on a friendly from header instead of the envelope from?

I just got done building up a multi-instance postfix set up that was supposed 
to divert mail from a certain domain to instance I had that applied some heavy 
throttling for a few to domains.

It turns out that their system using an envelope from that's the same for 
everyone, and just uses the customer specific from in the From: header. That's 
probably the better thing to be doing, but it just burnt me because the system 
I'm most recently working on matches aligns the envelope from and the header 
from.

Thanks in advance for any suggestions.

"Best" way to stop postfix from sending any DSN

[Sean Hennessey]

I'm doing some testing and am trying to figure out a way to set up postfix so 
that it won't ever send a DSN.

In googling around I found one option is to change the bounce service in 
master.cf to use the discard service. That seems to stop the DSN for a bounce 
to be sent out, but I get some warnings in the logs about attributes. I'm 
assuming that's because the discard service doesn't really know how to properly 
deal w/ a bounce request.

bounceunix  -   -   y   -   0   discard -v -v -v

Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: connection established
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: master_notify: status 0
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: deliver_request_initial: 
send initial status
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: send attr status = 0
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: vstream_fflush_some: fd 
12 flush 10
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: vstream_buf_get_ready: fd 
12 got 522
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: bounce socket: wanted 
attribute: flags
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: input attribute name: 
nrequest
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: warning: unexpected 
attribute nrequest from bounce socket (expecting: flags)
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: warning: 
deliver_request_get: error receiving common attributes
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: deliver_request_final: 
send: "" -1
Dec 30 23:15:59 OptiPlex-9010 postfix/discard[17341]: send attr status =

Is there a "correct" way to turn of all DSN notices for bounces?

RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

I already integrated into postfix. I just slapped that query into the .cf file 
and it "worked."
--
cat /etc/postfix/pgsql-transport.cf
hosts = 127.0.0.1

# The user name and password to log into the pgsql server.
user = XXX
password = XXX

# The database name on the servers.
dbname = postfixtransport

# Postfix 2.2 and later The SQL query template. See pgsql_table(5).
query = update send_counts set send_count = case when send_count > threshold 
then send_count else send_count + 1 end from domains, thresholds where 
send_counts.domain_id=domains.domain_id and 
send_counts.domain_id=thresholds.domain_id and domain_name='%s'; select 
transport_name as "transport" from matrix m inner join domains d on 
m.domain_id=d.domain_id inner join transport t on m.transport_id=t.transport_id 
left join send_counts s on m.domain_id=s.domain_id inner join thresholds h on 
m.domain_id=h.domain_id where domain_name='%s' and send_count <= threshold;

--

This also doesn't stop sending as when the thresholds are reached, the mail 
goes thru the other relay. So it stops mail flow thru one set of system, but 
the mail actually does flow out.

It's mimicking a system I've used before for slowly ramping up send volumes. 
Send 1000 emails a day to yahoo, msn, etc. for a few days, then 5000, then 
1, etc...

I've seen before when going full tilt with a never been seen before sending IP, 
Yahoo in particular, will shut you down pretty quick.

Right now this is all just POC and learning what might be possible.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Viktor Dukhovni
Sent: Wednesday, December 14, 2022 9:58 AM
To: postfix-users@postfix.org
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

On Wed, Dec 14, 2022 at 06:07:41AM +, Sean Hennessey wrote:
> Viktor and anyone else,
> 
> I'd like your opinion on something I've come up with that seems to 
> work in my test box. What I've done is set things up so that instead 
> of % thresholds I'm using a count of sent email. I fully expect the 
> counting to not be 100% accurate, off by a couple of tens or even 
> hundreds isn't a big deal. What I did was create the following tables:
>
> [elided]
>
> I'm using this for the query:
>
> query = update send_counts ...

How exactly do you imagine integrating such a query into Postfix?

[ Hint using this as a transport table may not do what you think
  it does.  Transport lookups happen per-recipient and in both
  smtpd(8) and qmgr(8) where lookups happen each time a message
  enters the active queue (and before actual delivery). ]

And is sending a small batch of messages and then stopping really a more 
effective approach to avoid problems due to lack of "reputation" with new new 
IPs?

-- 
Viktor.

Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

Viktor and anyone else,

I'd like your opinion on something I've come up with that seems to work in my 
test box. What I've done is set things up so that instead of % thresholds I'm 
using a count of sent email. I fully expect the counting to not be 100% 
accurate, off by a couple of tens or even hundreds isn't a big deal. What I did 
was create the following tables:
DROP TABLE IF EXISTS domains;
CREATE TABLE domains(
   domain_id INT GENERATED ALWAYS AS IDENTITY,
   domain_name text,
   PRIMARY KEY(domain_id)
);

DROP TABLE IF EXISTS transport;
CREATE TABLE transport(
   transport_id INT GENERATED ALWAYS AS IDENTITY,
   transport_name text,
   PRIMARY KEY(transport_id)
);

drop table if exists send_counts;
create table send_counts(
   domain_id int unique,
   send_count int,
   CONSTRAINT fk_domains
  FOREIGN KEY(domain_id)
    REFERENCES domains(domain_id)
    ON DELETE CASCADE
);

drop table if exists thresholds;
create table thresholds(
   domain_id int,
   threshold int,
   CONSTRAINT fk_domains
  FOREIGN KEY(domain_id)
    REFERENCES domains(domain_id)
    ON DELETE CASCADE
);


drop table if exists matrix;
CREATE TABLE matrix(
   matrix_id INT GENERATED ALWAYS AS IDENTITY,
   domain_id INT,
   transport_id INT,
   PRIMARY KEY(matrix_id),
   CONSTRAINT fk_domains
  FOREIGN KEY(domain_id)
    REFERENCES domains(domain_id)
    ON DELETE CASCADE,
   CONSTRAINT fk_transport
  FOREIGN KEY(transport_id)
    REFERENCES transport(transport_id)
    ON DELETE CASCADE
);

So  tables w/ the destination domains, transport to use for said domain, count 
of emails sent to said domains, thresholds for the domains, and matrix table to 
join the domains and transport. I went a bit overboard w/ normalizing the 
tables. It probably could have all been put into one, but I'm also using this 
an exercise to play w/ postgress. Once all the data has been loaded into the 
tables it's ready to roll. What I probably should do is use a user defined 
function, but I found that I can put an update and select in the .cfg file and 
it works. I'm using this for the query:
query = update send_counts set send_count = case when send_count > threshold 
then send_count else send_count + 1 end from domains, thresholds where 
send_counts.domain_id=domains.domain_id and 
send_counts.domain_id=thresholds.domain_id and domain_name='%s'; select 
transport_name as "transport" from matrix m inner join domains d on 
m.domain_id=d.domain_id inner join transport t on m.transport_id=t.transport_id 
left join send_counts s on m.domain_id=s.domain_id inner join thresholds h on 
m.domain_id=h.domain_id where domain_name='%s' and send_count <= threshold;

There are some extra fields and joins in that select query as I was using it 
for testing stuff.

Here's what my test data looks like:
select domain_name,transport_name,threshold,send_count from matrix m inner join 
domains d on m.domain_id=d.domain_id inner join transport t on 
m.transport_id=t.transport_id left join send_counts s on 
m.domain_id=s.domain_id inner join thresholds h on m.domain_id=h.domain_id;
 domain_name |transport_name | threshold | send_count
-+---+---+
 gmail.com   | relay:[send.blah.com] |15 | 16
 yahoo.com   | relay:[send.blah.com] |25 | 20
(2 rows)

The small time test I did worked like a charm, but I'm only doing a single 
client sending, not at scale.

I'm interested in anyone's thoughts on what I might have overlooked that could 
come back to haunt me if I decide to roll this out anymore.

Thanks in advance.
Sean


-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Sean Hennessey
Sent: Wednesday, November 30, 2022 11:38 PM
To: postfix-users@postfix.org
Subject: RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

Viktor,

I want to thank you a million for this. I finally read up on the docs and got 
this working. I'm still going to do some more in depth testing, but my quick 
little testing seems to be doing exactly what I wanted.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Viktor Dukhovni
Sent: Tuesday, November 29, 2022 3:44 AM
To: postfix-users@postfix.org
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

On Mon, Nov 28, 2022 at 08:57:37PM +, Sean Hennessey wrote:

> I searched the list archives and saw the thread of gradual shift of
> traffic from back in February of this year. That gives me some ideas,
> but that seems to be for all traffic, not a subset.
>
> I'd really like a way to send X% of gmail.com traffic to one relay and
> the rest to another relay. Ditto for a couple of other major ESP's
> like Yahoo, MS, etc...

If you're willing to spin up a small Postgres da

RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

Thanks. Can you point me to docs or what to search for to set up the per-sender 
default transports? I might be able to make that work. Running multiple postfix 
instances isn't a show stopper for me for this exercise.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Viktor Dukhovni
Sent: Thursday, December 8, 2022 11:15 AM
To: postfix-users@postfix.org
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

On Thu, Dec 08, 2022 at 04:01:26PM +, Sean Hennessey wrote:

> As is sometimes the case, this has turned a bit into going down a 
> rabbit hole. I find the case now that it would be "really nice" to be 
> able to also throw from addresses in the loop here. So basically, if 
> the from address is a...@somewhere.com, then I want to relay X% of 
> traffic to one relay as I've got working below. I looked at the 
> pgsql_table man page, but it looks like it only takes in the to 
> address for the %s for the transport map. I'm assuming there is other 
> maps I can do for the from address, but I'm not sure if it's possible 
> to create something that chains them together. I'm going to dive into 
> the postfix.org docs to see if anything pops for me, but I figured I'd 
> throw this out there in the hopes that better minds then I might have 
> some ideas.

Sorry you can't chain lookups on sender *and* recipient addresses (more 
generally multiple input keys), and to avoid lots of possible mistakes, 
probably shouldn't anyway.

What you can do with sender addresses is configure per-sender *default* 
transports, but these only apply when the recipient is for a remote domain, and 
has no explicit recipient-specific transport setting.

The only way to chain these is multiple Postfix instances, where the default 
transport for some recipients sends mail to separate Postfix instances that are 
warming up some recipient domains, while all other senders are delivered 
normally (direct-to-MX with no transport overrides).

-- 
Viktor.

RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

As is sometimes the case, this has turned a bit into going down a rabbit hole. 
I find the case now that it would be "really nice" to be able to also throw 
from addresses in the loop here. So basically, if the from address is 
a...@somewhere.com, then I want to relay X% of traffic to one relay as I've got 
working below. I looked at the pgsql_table man page, but it looks like it only 
takes in the to address for the %s for the transport map. I'm assuming there is 
other maps I can do for the from address, but I'm not sure if it's possible to 
create something that chains them together. I'm going to dive into the 
postfix.org docs to see if anything pops for me, but I figured I'd throw this 
out there in the hopes that better minds then I might have some ideas.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Sean Hennessey
Sent: Wednesday, November 30, 2022 11:38 PM
To: postfix-users@postfix.org
Subject: RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

Viktor,

I want to thank you a million for this. I finally read up on the docs and got 
this working. I'm still going to do some more in depth testing, but my quick 
little testing seems to be doing exactly what I wanted.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Viktor Dukhovni
Sent: Tuesday, November 29, 2022 3:44 AM
To: postfix-users@postfix.org
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

On Mon, Nov 28, 2022 at 08:57:37PM +, Sean Hennessey wrote:

> I searched the list archives and saw the thread of gradual shift of 
> traffic from back in February of this year. That gives me some ideas, 
> but that seems to be for all traffic, not a subset.
> 
> I'd really like a way to send X% of gmail.com traffic to one relay and 
> the rest to another relay. Ditto for a couple of other major ESP's 
> like Yahoo, MS, etc...

If you're willing to spin up a small Postgres database (modulo typos on my part 
that should be easy to correct):

query = SELECT U."transport"
FROM (
SELECT CASE WHEN floor(random()*100) <= T."weight"
   THEN T."transport"
   END AS "transport"
FROM "transports" AS T
WHERE T."domain" = '%s'
) AS U
WHERE U."transport" IS NOT NULL;

Just populate a table:

CREATE TABLE IF NOT EXISTS "transports" (
"domain" TEXT PRIMARY KEY,
"transport" TEXT NOT NULL,
"weight" INTEGER NOT NULL
);
INSERT INTO "transports" ("domain", "transport", "weight")
VALUES ( "gmail.com", "relay:[gmail-relay.example]", 99),
   ( "yahoo.com", "relay:[yahoo-relay.example]", 50),
   ... ;

And gradually lower the weights until, at weight 0, 99% of the traffic is 
direct to MX and just 1% of the traffic goes to the bypass relay and after that 
the row can be deleted.  Initially, at weight 99, all the traffic goes to the 
bypass relay.

If you want to specify a custom transport even after removing the relay, add a 
fourth (nullable) column and use that value in an ELSE clause of the CASE 
statement, in which case that value will be used when the bypass is not 
selected.

Keeping the Postgres database local to the MTA will improve performance and 
reliability.  I'd resist the temptation to centralise it, but that is an option 
if you're willing to have Postfix stall when a remote DB server is unreachable 
or slow, or can somehow avoid that.

It is not obvious to me, just at the moment, how to do this with the built-in 
Postfix randmap, pipemap, uniomap, ...

-- 
Viktor.

Is it possible to rewrite the envelop from address for all mail that's being relayed?

[Sean Hennessey]

I'm curios if it's possible to set up postfix to rewrite the envelop from 
address for all email the instance is relaying? I'm doing some research and 
want to force the envelop from to be a known value that I know will pass spf 
checks.

Thanks in advance.

Sean

RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

Viktor,

I want to thank you a million for this. I finally read up on the docs and got 
this working. I'm still going to do some more in depth testing, but my quick 
little testing seems to be doing exactly what I wanted.

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Viktor Dukhovni
Sent: Tuesday, November 29, 2022 3:44 AM
To: postfix-users@postfix.org
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

On Mon, Nov 28, 2022 at 08:57:37PM +, Sean Hennessey wrote:

> I searched the list archives and saw the thread of gradual shift of 
> traffic from back in February of this year. That gives me some ideas, 
> but that seems to be for all traffic, not a subset.
> 
> I'd really like a way to send X% of gmail.com traffic to one relay and 
> the rest to another relay. Ditto for a couple of other major ESP's 
> like Yahoo, MS, etc...

If you're willing to spin up a small Postgres database (modulo typos on my part 
that should be easy to correct):

query = SELECT U."transport"
FROM (
SELECT CASE WHEN floor(random()*100) <= T."weight"
   THEN T."transport"
   END AS "transport"
FROM "transports" AS T
WHERE T."domain" = '%s'
) AS U
WHERE U."transport" IS NOT NULL;

Just populate a table:

CREATE TABLE IF NOT EXISTS "transports" (
"domain" TEXT PRIMARY KEY,
"transport" TEXT NOT NULL,
"weight" INTEGER NOT NULL
);
INSERT INTO "transports" ("domain", "transport", "weight")
VALUES ( "gmail.com", "relay:[gmail-relay.example]", 99),
   ( "yahoo.com", "relay:[yahoo-relay.example]", 50),
   ... ;

And gradually lower the weights until, at weight 0, 99% of the traffic is 
direct to MX and just 1% of the traffic goes to the bypass relay and after that 
the row can be deleted.  Initially, at weight 99, all the traffic goes to the 
bypass relay.

If you want to specify a custom transport even after removing the relay, add a 
fourth (nullable) column and use that value in an ELSE clause of the CASE 
statement, in which case that value will be used when the bypass is not 
selected.

Keeping the Postgres database local to the MTA will improve performance and 
reliability.  I'd resist the temptation to centralise it, but that is an option 
if you're willing to have Postfix stall when a remote DB server is unreachable 
or slow, or can somehow avoid that.

It is not obvious to me, just at the moment, how to do this with the built-in 
Postfix randmap, pipemap, uniomap, ...

-- 
Viktor.

Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

I'm going to toy w/ that one tomorrow, I don't recall seeing in your original 
post though the plumbing to config postfix to use the postgress db. I was going 
to send a mail on that tomorrow, but while you are here 😉




From: owner-postfix-us...@postfix.org  on 
behalf of Viktor Dukhovni 
Sent: Wednesday, November 30, 2022 1:34 AM
To: postfix-users@postfix.org 
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

On Wed, Nov 30, 2022 at 05:10:26AM +, Sean Hennessey wrote:
> I've been poking at this a bit tonight. I am by no means a postfix expert. In 
> the hopes that someone can see an obvious configuration issue, I'm going to 
> post what I'm seeing:
>
> In my main.cf;
> smtpd_recipient_restrictions =  check_recipient_access
>   inline:{
>  { gmail.com = class-gmail }
>  { yahoo.com = class-yahoo }
>   } permit_mynetworks permit_sasl_authenticated defer_unauth_destination
>
> restriction_classes = class-gmail, class-yahoo

s/restriction_classes/smtpd_restriction_classes/

For every parameter you configure, you should check the postconf(5)
documentation to try to understand whether you've at least typed the
name correctly, and whether the setting makes plausible sense.

> class-gmail = check_recipient_access randmap:{filter smtp1:, filter smtp2:}
> class-yahoo = check_recipient_access randmap:{filter smtp1:, filter smtp2:}
>
> postconf complains about this:
> postconf: warning: /etc/postfix/main.cf: unused parameter: 
> class-yahoo=check_recipient_access randmap:{filter smtp1:, filter smtp2:}
> postconf: warning: /etc/postfix/main.cf: unused parameter: 
> class-gmail=check_recipient_access randmap:{filter smtp1:, filter smtp2:}
> postconf: warning: /etc/postfix/main.cf: unused parameter: 
> restriction_classes=class-gmail, class-yahoo

This is a clear sign that the parameter name is not correct.

FWIW, my advice remains to consider the Postgres approach, it is more
correct and scales to more domains.  Yes, it requires some effort to set
up a minimal Postgres database.

--
Viktor.

[ My SQL "INSERT" statement incorrectly used double-quotes instead of
  single-quotes for string values in the "VALUES" clause.  In Postgres
  double-quotes as for schema names, and single-quotes are for string
  data.  Other than that, the recipe seems correct. ]

Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

Another one that bounced.


From: Sean Hennessey 
Sent: Wednesday, November 30, 2022 1:14 AM
To: Postfix users 
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

So final update for the night.

It's got to be the way it's parsing the randmap:{filter smtp1:send.smtp.com:25, 
filter smtp2:}
That's being parsed as four things, not two. Two "filter", one 
"smtp1:send.smtp.com:25", and one "smtp2"

Nov 30 01:10:11 OptiPlex-9010 postfix/smtpd[504846]: maps_find: randmap:{filter 
smtp1:send.smtp.com:25, filter smtp2:}: randmap:{filter smtp1:send.smtp.com:25, 
filter smtp2:}(0,lock|fold_fix|utf8_request): x...@gmail.com = 
smtp1:send.smtp.com:25
Nov 30 01:10:24 OptiPlex-9010 postfix/smtpd[504846]: maps_find: randmap:{filter 
smtp1:send.smtp.com:25, filter smtp2:}: randmap:{filter smtp1:send.smtp.com:25, 
filter smtp2:}(0,lock|fold_fix|utf8_request): x...@gmail.com = filter
Nov 30 01:10:34 OptiPlex-9010 postfix/smtpd[504846]: maps_find: randmap:{filter 
smtp1:send.smtp.com:25, filter smtp2:}: randmap:{filter smtp1:send.smtp.com:25, 
filter smtp2:}(0,lock|fold_fix|utf8_request): x...@gmail.com = smtp2

So now just have to figure out how to get it to parse that as two filter 
actions.

Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

I tried sending this earlier, but it bounced back because it was too long. let 
me trim it and try again.


From: Sean Hennessey 
Sent: Wednesday, November 30, 2022 12:49 AM
To: Postfix users 
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

Some more information...

Updated main.cf:
smtpd_relay_restrictions =  check_recipient_access
  inline:{
 { gmail.com = class-gmail }
 { yahoo.com = class-yahoo }
  } permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_recipient_restrictions =  permit_mynetworks permit_sasl_authenticated 
defer_unauth_destination

smtpd_restriction_classes = class-gmail, class-yahoo
class-gmail = check_recipient_access randmap:{filter smtp1:, filter 
smtp2:send.smtp.com:25}
class-yahoo = check_recipient_access randmap:{filter smtp1:, filter smtp2:

Now when I send emails one goes fine, and the other gets the error, so one 
filter is good the other not so much:
Working:
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: check_mail_access: 
x...@gmail.com
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: ctable_locate: leave 
existing entry key sean.hennessey@YYY?x...@gmail.com
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: maps_find: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  }: x...@gmail.com: 
not found
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: maps_find: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  }: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  
}(0,lock|fold_fix|utf8_request): gmail.com = class-gmail
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: mail_addr_find: 
x...@gmail.com -> class-gmail
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: check_table_result: 
inline:{ { gmail.com = class-gmail } { yahoo.com = class-yahoo }  } 
class-gmail x...@gmail.com
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: >>> START Recipient 
address RESTRICTIONS <<<
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=class-gmail
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: >>> START Recipient 
address RESTRICTIONS <<<
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=check_recipient_access
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: check_mail_access: 
x...@gmail.com
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: ctable_locate: leave 
existing entry key sean.hennessey@YYY?x...@gmail.com
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: maps_find: randmap:{filter 
smtp1:, filter smtp2:send.smtp.com:25}: randmap:{filter smtp1:, filter 
smtp2:send.smtp.com:25}(0,lock|fold_fix|utf8_request): x...@gmail.com = filter
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: mail_addr_find: 
x...@gmail.com -> filter
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: check_table_result: 
randmap:{filter smtp1:, filter smtp2:send.smtp.com:25} filter x...@gmail.com
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: warning: access table 
randmap:{filter smtp1:, filter smtp2:send.smtp.com:25} entry "x...@gmail.com" 
has FILTER entry without value
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=check_recipient_access status=0
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: >>> END Recipient address 
RESTRICTIONS <<<
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=class-gmail status=0
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: >>> END Recipient address 
RESTRICTIONS <<<
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=check_recipient_access status=0
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=permit_mynetworks
Nov 30 00:33:25 OptiPlex-9010 postfix/smtpd[503381]: permit_mynetworks: 
Latitude-E5540.lan 192.168.0.141

Not working:
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: >>> START Recipient 
address RESTRICTIONS <<<
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: generic_checks: 
name=check_recipient_access
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: check_mail_access: 
x...@gmail.com
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: ctable_locate: leave 
existing entry key sean.hennessey@YYY?x...@gmail.com
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: maps_find: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  }: x...@gmail.com: 
not found
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: maps_find: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  }: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  
}(0,lock|fold_fix|utf8_request): gmail.com = class-gmail
Nov 30 00:33:26 OptiPlex-9010 postfix/smtpd[503381]: mail_addr_find: 
x...@gmail.com -> class-gmail
Nov 30 00:3

Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

sigh​

Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: >>> START Recipient 
address RESTRICTIONS <<<
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: generic_checks: 
name=check_recipient_access
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: check_mail_access: 
x...@gmail.com
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: ctable_locate: leave 
existing entry key sean.hennessey@YYY?x...@gmail.com
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: maps_find: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  }: x...@gmail.com: 
not found
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: maps_find: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  }: inline:{ { 
gmail.com = class-gmail } { yahoo.com = class-yahoo }  
}(0,lock|fold_fix|utf8_request): gmail.com = class-gmail
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: mail_addr_find: 
x...@gmail.com -> class-gmail
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: check_table_result: 
inline:{ { gmail.com = class-gmail } { yahoo.com = class-yahoo }  } 
class-gmail x...@gmail.com
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: >>> START Recipient 
address RESTRICTIONS <<<
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: generic_checks: 
name=class-gmail
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: >>> START Recipient 
address RESTRICTIONS <<<
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: generic_checks: 
name=check_recipient_access
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: check_mail_access: 
x...@gmail.com
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: ctable_locate: leave 
existing entry key sean.hennessey@YYY?x...@gmail.com
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: maps_find: randmap:{ 
{filter smtp1:send.smtp.com:25}, {filter smtp2:} }: randmap:{ {filter 
smtp1:send.smtp.com:25}, {filter smtp2:} }(0,lock|fold_fix|utf8_request): 
x...@gmail.com = {filter smtp2:}
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: mail_addr_find: 
x...@gmail.com -> {filter smtp2:}
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: check_table_result: 
randmap:{ {filter smtp1:send.smtp.com:25}, {filter smtp2:} } {filter smtp2:} 
x...@gmail.com
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: warning: access table 
randmap:{ {filter smtp1:send.smtp.com:25}, {filter smtp2:} } has entry with 
lookup table: {filter smtp2:}
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: warning: do not specify 
lookup tables inside SMTPD access maps
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: warning: define a 
restriction class and specify its name instead.
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: NOQUEUE: reject: RCPT from 
Latitude-E5540.lan[192.168.0.141]: 451 4.3.5 Server configuration error; 
from= to= proto=SMTP helo=
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: > 
Latitude-E5540.lan[192.168.0.141]: 451 4.3.5 Server configuration error
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: smtp_get: EOF
Nov 30 01:15:53 OptiPlex-9010 postfix/smtpd[505479]: match_hostname: smtp

Hopefully I'll have better luck tomorrow.

Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

CTIONS <<<
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: generic_checks: 
name=check_recipient_access
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: check_mail_access: 
x...@gmail.com
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: ctable_locate: leave 
existing entry key sean.hennessey@YYY?x...@gmail.com
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: maps_find: randmap:{filter 
smtp1:, filter smtp2:}: randmap:{filter smtp1:, filter 
smtp2:}(0,lock|fold_fix|utf8_request): x...@gmail.com = smtp2:
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: mail_addr_find: 
x...@gmail.com -> smtp2:
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: check_table_result: 
randmap:{filter smtp1:, filter smtp2:} smtp2: x...@gmail.com
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: warning: access table 
randmap:{filter smtp1:, filter smtp2:} has entry with lookup table: smtp2:
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: warning: do not specify 
lookup tables inside SMTPD access maps
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: warning: define a 
restriction class and specify its name instead.
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: NOQUEUE: reject: RCPT from 
Latitude-E5540.lan[192.168.0.141]: 451 4.3.5 Server configuration error; 
from= to= proto=SMTP helo=
Nov 30 00:03:37 OptiPlex-9010 postfix/smtpd[501383]: > 
Latitude-E5540.lan[192.168.0.141]: 451 4.3.5 Server configuration error

I'm going to try to dig into more details on the morrow, but in case anyone 
happens to know.

Thanks in advance.

From: owner-postfix-us...@postfix.org  on 
behalf of Wietse Venema 
Sent: Tuesday, November 29, 2022 9:59 AM
To: Postfix users 
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

>Sean Hennessey:
> All,
>
> I'm bringing a new sending IP online and need to know if there is
> an easy way to warm it up w/ Postfix.
>
> For those that don't know, warming up is a process where you start
> to send small amounts of mail of the new IP till it's built up a
> good enough reputation.
>
> In a past life I use Port25's powermta software. It has an easy
> config way of doing this where you can say for gmail.com divert
> the first X emails to use this way of sending instead of the normal
> way.
>
> I'm trying to see if there is something similar in postfix.
>
> So basically I need a way to take a configurable amount of mail
> for various domains and send them on smtp endpoint instead of the
> relay that is currently configured.
>
> I searched the list archives and saw the thread of gradual shift
> of traffic from back in February of this year. That gives me some
> ideas, but that seems to be for all traffic, not a subset.
>
> I'd really like a way to send X% of gmail.com traffic to one relay
> and the rest to another relay. Ditto for a couple of other major
> ESP's like Yahoo, MS, etc...
>
> Thanks in advance.

The two examples below support different old/new IP address ratios
for different destinations, as well as the same ratios for all
destinations.

To change the ratios over time, you would have to update main.cf
and rely on Postfix built-in behavior to restart Postfix SMTP server
processes after 100 SMTP client connections.

First, specify master.cf entries with different source IP addresses.
Each entry uses the proper smtp_helo_name that matches the PTR value
for the smtp_bind_address setting.

In /etc/postfix/master.cf:
==

smtp1 unix  -   -   n   -   -   smtp
  -o { smtp_bind_address = 1.2.3.1 }
  -o { smtp_helo_name = helo-for-1.2.3.1 }

smtp2 unix  -   -   n   -   -   smtp
  -o { smtp_bind_address = 1.2.3.2 }
  -o { smtp_helo_name = helo-for-1.2.3.2 }

In /etc/postfix/main.cf:


Next an example that supports different warming up ratios for
different destinations. See "Notes" below for a simpler approach
when the warming up ratios can be the same for all destinations.

This example uses a nested table, which requires a restriction class.
smtpd_recipient_restrictions = check_recipient_access
  inline:{
 { gmail.com = class-gmail }
 { yahoo.com = class-yahoo }
  }
  ...other restrictions...

  # To get (90%, 10%), specify one filter 9x and the other filter 1x.
  # The order does not matter.
  restriction_classes = class-gmail, class-yahoo
  class-gmail = check_recipient_access randmap:{filter smtp1:, filter smtp2:}
  class-yahoo = check_recipient_access randmap:{filter smtp1:, filter smtp2:}

Notes:
==

- Instead of inline:{...}, a hash or pcre table could work too.
  With pcre you can easily specify a catch-all rule at the end.

- If you can use the same randmap:{} settings for all destinations, then
  you can avoid the need for restriction_classes and for

RE: Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

Wietse,

Thanks a bunch for this. I'm going to try to play w/ this on a test box to see 
if I can get it working. It looks detailed enough here.

Sean

-Original Message-
From: owner-postfix-us...@postfix.org  On 
Behalf Of Wietse Venema
Sent: Tuesday, November 29, 2022 10:00 AM
To: Postfix users 
Subject: Re: Is there an easy way to "warm up" a new sending IP w/ Postfix

>Sean Hennessey:
> All,
>
> I'm bringing a new sending IP online and need to know if there is an 
> easy way to warm it up w/ Postfix.
>
> For those that don't know, warming up is a process where you start to 
> send small amounts of mail of the new IP till it's built up a good 
> enough reputation.
>
> In a past life I use Port25's powermta software. It has an easy config 
> way of doing this where you can say for gmail.com divert the first X 
> emails to use this way of sending instead of the normal way.
>
> I'm trying to see if there is something similar in postfix.
>
> So basically I need a way to take a configurable amount of mail for 
> various domains and send them on smtp endpoint instead of the relay 
> that is currently configured.
>
> I searched the list archives and saw the thread of gradual shift of 
> traffic from back in February of this year. That gives me some ideas, 
> but that seems to be for all traffic, not a subset.
>
> I'd really like a way to send X% of gmail.com traffic to one relay and 
> the rest to another relay. Ditto for a couple of other major ESP's 
> like Yahoo, MS, etc...
>
> Thanks in advance.

The two examples below support different old/new IP address ratios for 
different destinations, as well as the same ratios for all destinations.

To change the ratios over time, you would have to update main.cf and rely on 
Postfix built-in behavior to restart Postfix SMTP server processes after 100 
SMTP client connections.

First, specify master.cf entries with different source IP addresses.
Each entry uses the proper smtp_helo_name that matches the PTR value for the 
smtp_bind_address setting.

In /etc/postfix/master.cf:
==

smtp1 unix  -   -   n   -   -   smtp
  -o { smtp_bind_address = 1.2.3.1 }
  -o { smtp_helo_name = helo-for-1.2.3.1 }

smtp2 unix  -   -   n   -   -   smtp
  -o { smtp_bind_address = 1.2.3.2 }
  -o { smtp_helo_name = helo-for-1.2.3.2 }

In /etc/postfix/main.cf:


Next an example that supports different warming up ratios for different 
destinations. See "Notes" below for a simpler approach when the warming up 
ratios can be the same for all destinations.

This example uses a nested table, which requires a restriction class.
smtpd_recipient_restrictions = check_recipient_access
  inline:{
 { gmail.com = class-gmail }
 { yahoo.com = class-yahoo }
  }
  ...other restrictions...

  # To get (90%, 10%), specify one filter 9x and the other filter 1x.
  # The order does not matter.
  restriction_classes = class-gmail, class-yahoo
  class-gmail = check_recipient_access randmap:{filter smtp1:, filter smtp2:}
  class-yahoo = check_recipient_access randmap:{filter smtp1:, filter smtp2:}

Notes:
==

- Instead of inline:{...}, a hash or pcre table could work too. 
  With pcre you can easily specify a catch-all rule at the end.

- If you can use the same randmap:{} settings for all destinations, then
  you can avoid the need for restriction_classes and for the lookup
  table that returns restriction class names. Just select the filter
  action with:

  smtpd_recipient_restrictions =
 check_recipient_access randmap:{filter smtp1:, filter smtp2:}
 ...other restrictions...

- For the future: we could make randmap smarter and support a weight
  factor, like randmap:{{filter smtp1:}*99, {filter smtp2:}*1} where
  a weight of *1 is of course the default.

The examples were not tested...

Wietse

Is there an easy way to "warm up" a new sending IP w/ Postfix

[Sean Hennessey]

All,

I'm bringing a new sending IP online and need to know if there is an easy way 
to warm it up w/ Postfix.

For those that don't know, warming up is a process where you start to send 
small amounts of mail of the new IP till it's built up a good enough reputation.

In a past life I use Port25's powermta software. It has an easy  config way of 
doing this where you can say for gmail.com divert the first X emails to use 
this way of sending instead of the normal way.

I'm trying to see if there is something similar in postfix.

So basically I need a way to take a configurable amount of mail for various 
domains and send them on smtp endpoint instead of the relay that is currently 
configured.

I searched the list archives and saw the thread of gradual shift of traffic 
from back in February of this year. That gives me some ideas, but that seems to 
be for all traffic, not a subset.

I'd really like a way to send X% of gmail.com traffic to one relay and the rest 
to another relay. Ditto for a couple of other major ESP's like Yahoo, MS, etc...

Thanks in advance.

Sean