Re: [CentOS] Postfix and virtual mail boxes.[SOLVED - kinda]
On 11/10/21 2:06 pm, Rob Kampen wrote: OK - just to let you know the munge I used. example.com is an alias domain for example.org which is the actual domain with Maildir space on the server. rob@ is alias for rkampen@ thus the only real address is rkam...@example.org That's not what I meant by munge, but it helps. Anyhow, what you showed me is what I need to see. now the results This is all good, except... [root@mx rkampen]# postmap -q @example.com mysql:/etc/postfix/mysql-virtual_forwardings.cf @example.org This is a bad idea. It's not the cause of your problem (as you discovered below) but it will lead you to become a backscatter source. I'll explain. Postfix checks virtual_alias_maps entries twice, once in smtpd and once in cleanup (actually trivial_rewrite, which is called from cleanup). At the smtpd stage it simply checks to make sure that there is a virtual_alias_map entry for the recipient address, but it doesn't care where that entry points, then (after other checks) postfix accepts and queues the message. The message is then passed to cleanup, which, as once of its functions, calls trivial_rewrite which, once again, checks virtual_alias_maps and in turn rewrites the recipient to the target email address. (It then loops and does this until there are no more rewrites to be done). Then delivery is attempted to the target address. So in your case, what happens when the original recipient is "nonexist...@example.com"? Well Postfix first checks virtual_alias_maps for entries of "nonexist...@example.com", "nonexistent" and "@example.com", it finds a match for the last of these and so the message passes the checks in smtpd and the message is accepted for delivery and queued. Then the message is passed to cleanup which checks with trivial_rewrite and trivial_rewrite checks virtual_alias_maps (again) and after finding the "@example.com" entry it rewrites the recipient to "nonexist...@example.org". Then Postfix attempts to deliver the message to nonexist...@example.org, but since there's no such recipient and since postfix has already accepted the message for delivery it generates a bounce message which it sends to the envelope sender. Thus you have a situation where messages can be sent to invalid recipients and your system will bounce them, and as such you become a source of backscatter. You can confirm this by attempting to send a message to nonexist...@example.com and another one to nonexist...@example.org. The example.com one will generate a bounce from Postfix while the example.org one will reject the message instead (you'll be able to see the difference in your Postfix logs). The solution here is to not use an @example.com entry in virtual_alias_maps, but instead to have a separate entry for each valid recipient in example.com. Since you're using mysql it is possible to craft a mysql query that can check the valid example.org recipients and only return example.com matches for those recipients, I'll leave that as an exorsize for you, but if you have problems with it I'll be happy to help. Well that may have done it! Now I get a correctly sent email with the alias substitutions done. Funny how that line seems to cause no error on my two original MX - looks like I better check them out a little more too. That's great. So basically what you've done by removing that entry is to bypass the policyd-spf check from Postfix. What that means is that the problem is in policyd-spf and this is further confirmed by the following log entry: Oct 11 13:53:10 mx policyd-spf[10723]: ERROR: Unknown name "TestOnly" in file "/etc/python-policyd-spf/policyd-spf.conf" Unfortunately I am not a policyd-spf expert by any stretch, so at this stage I'll direct you to their community support for further help (unless someone else here wants to chime in and help). What I can say is that if you fix your policyd-spf issue then it is likely that you can add that entry back into the Postfix configuration and it will work. Thanks Peter, your help has been invaluable and MUCH appreciated! You're quite welcome. Peter ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Postfix and virtual mail boxes.[SOLVED - kinda]
On 11/10/21 12:53 am, Peter wrote: On 10/10/21 11:28 pm, Rob Kampen wrote: smtp inet n - n - - smtpd -o smtpd_recipient_restrictions= -o content_filter=spamassassin I assume based on what you've said before that this is after you added the workaround you mentioned, but the logs below are without the smtpd_recipient_restrictions= part here? Correct, once I added the -o smtpd_recipient_restrictions= the alias substitutions worked and the log becomes much longer as all the various processes complete and add their trace to the maillog. Cannot see how this log listing can possibly help as it contains only three lines Nonetheless I do appreciate seeing them, no offense but you can never tell if someone's interpretations of the logs are accurate and so it's best just to see the logs themselves. Here is the log of the incorrectly rejected email coming into the new MX - very short as it immediately rejects the alias recipient address - which my other two MX do not do. Right. This led me to the conclusion that the alias substitution is not taking place on my new MX whereas it does on my two working MX - hence my addition to the smtp processing line at the top of the master.cf file. I wouldn't jump to that conclusion just yet, though. That said, based on your config and logs I think I may have been wrong in my previous guess and it may very well be related to your policyd-spf. More on that in a bit. Can you provide the output of the following commands (but substitute the actual recipient domain and address for the munged versions you supplied here): postmap -q example.com mysql:/etc/postfix/mysql-virtual_alias_domains.cf postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_forwardings.cf postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_email2email.cf postmap -q example.com mysql:/etc/postfix/mysql-virtual_domains.cf postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_mailboxes.cf The results of the above should give a much better picture of what's going on. OK - just to let you know the munge I used. example.com is an alias domain for example.org which is the actual domain with Maildir space on the server. rob@ is alias for rkampen@ thus the only real address is rkam...@example.org now the results [root@mx rkampen]# postmap -q example.org mysql:/etc/postfix/mysql-virtual_alias_domains.cf [root@mx rkampen]# postmap -q example.org mysql:/etc/postfix/mysql-virtual_domains.cf example.org [root@mx rkampen]# postmap -q example.com mysql:/etc/postfix/mysql-virtual_alias_domains.cf example.com [root@mx rkampen]# postmap -q example.com mysql:/etc/postfix/mysql-virtual_domains.cf [root@mx rkampen]# postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_forwardings.cf [root@mx rkampen]# postmap -q r...@example.org mysql:/etc/postfix/mysql-virtual_forwardings.cf rkam...@example.org [root@mx rkampen]# postmap -q @example.com mysql:/etc/postfix/mysql-virtual_forwardings.cf @example.org [root@mx rkampen]# postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_email2email.cf [root@mx rkampen]# postmap -q r...@example.org mysql:/etc/postfix/mysql-virtual_email2email.cf [root@mx rkampen]# postmap -q rkam...@example.org mysql:/etc/postfix/mysql-virtual_email2email.cf rkam...@example.org [root@mx rkampen]# postmap -q rkam...@example.com mysql:/etc/postfix/mysql-virtual_email2email.cf [root@mx rkampen]# postmap -q rkam...@example.com mysql:/etc/postfix/mysql-virtual_mailboxes.cf [root@mx rkampen]# postmap -q rkam...@example.org mysql:/etc/postfix/mysql-virtual_mailboxes.cf example.org/rkampen/ As all but mysql-virtual_alias_domains.cf are copies from the other MX, I think these are fine. Also as email presented via port 587 via an authenticated STARTTLS session actually work fine, I have no reason to suspect any issues in this area. To check if it's the policyd that's causing the problem can you modify the smtpd_recipient_restrictions line in main.cf and remove just the "check_policy_service inet:localhost:12350," part? So that it reads something like: smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policyd-spf Then check to see if it works after that (and provide logs again so I can check things over). Note this also means reverting your workaround in master.cf for this test. Well that may have done it! Now I get a correctly sent email with the alias substitutions done. Funny how that line seems to cause no error on my two original MX - looks like I better check them out a little more too. Here is the munged log (same munging as above) Oct 11 13:53:09 mx postfix/smtpd[10711]: connect from mail-pj1-x1030.google.com[2607:f8b0:4864:20::1030] Oct 11 13:53:10 mx policyd-spf[10723]: ERROR: Unknown name "TestOnly" in file "/etc/python-policyd-spf/policyd-spf.conf"
Re: [CentOS] Postfix and virtual mail boxes.[SOLVED - kinda]
On 10/10/21 11:28 pm, Rob Kampen wrote: smtp inet n - n - - smtpd -o smtpd_recipient_restrictions= -o content_filter=spamassassin I assume based on what you've said before that this is after you added the workaround you mentioned, but the logs below are without the smtpd_recipient_restrictions= part here? Cannot see how this log listing can possibly help as it contains only three lines Nonetheless I do appreciate seeing them, no offense but you can never tell if someone's interpretations of the logs are accurate and so it's best just to see the logs themselves. Here is the log of the incorrectly rejected email coming into the new MX - very short as it immediately rejects the alias recipient address - which my other two MX do not do. Right. This led me to the conclusion that the alias substitution is not taking place on my new MX whereas it does on my two working MX - hence my addition to the smtp processing line at the top of the master.cf file. I wouldn't jump to that conclusion just yet, though. That said, based on your config and logs I think I may have been wrong in my previous guess and it may very well be related to your policyd-spf. More on that in a bit. Can you provide the output of the following commands (but substitute the actual recipient domain and address for the munged versions you supplied here): postmap -q example.com mysql:/etc/postfix/mysql-virtual_alias_domains.cf postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_forwardings.cf postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_email2email.cf postmap -q example.com mysql:/etc/postfix/mysql-virtual_domains.cf postmap -q r...@example.com mysql:/etc/postfix/mysql-virtual_mailboxes.cf The results of the above should give a much better picture of what's going on. To check if it's the policyd that's causing the problem can you modify the smtpd_recipient_restrictions line in main.cf and remove just the "check_policy_service inet:localhost:12350," part? So that it reads something like: smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policyd-spf Then check to see if it works after that (and provide logs again so I can check things over). Note this also means reverting your workaround in master.cf for this test. Peter ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Postfix and virtual mail boxes.[SOLVED - kinda]
On 9/10/21 9:55 pm, Peter wrote: On 9/10/21 12:26 pm, Rob Kampen wrote: So, after many dozens of hours and sending test emails I have found a solution (work around) that appears to work okay. It is now different to the original two MX servers I cloned from, in that the maillog shows a different cycle of processing, and it now fails a truly unknown mailbox much later in the process - thus higher workload on my MX. But the key thing is that it does now do the virtual_alias checks on incoming emails on port 25 before rejecting. if your MX is not rejecting messages to invalid recipients right away but instead bounces the messages later on you become a backscatter source (See https://www.backscatterer.org/?target=bounces). Understood. On the two existing MX the recipient checks happen up front AFTER alias substitutions and hence no reject of valid email addresses. I have been unable to achieve this behaviour with the new MX your server needs a properly configured list of valid recipients so it knows right away what recipients to accept and which ones to reject. Agrred, and it has - in mysql tables. No idea why this third MX is behaving differently. It has a dual stack IP, so I disabled IPv6 access and tried again, but that certainly wasn't the cause of the difference in processing. If you can provide the output of the following two commands it would be very helpful in troubleshooting your problem: postconf -nf alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases broken_sasl_auth_clients = yes command_directory = /usr/sbin config_directory = /etc/postfix content_filter = smtp-amavis:[127.0.0.1]:10024 daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 dovecot_destination_recipient_limit = 1 html_directory = no inet_interfaces = all inet_protocols = all mail_owner = postfix mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man message_size_limit = 3072 milter_default_action = accept mydestination = localhost localhost.localdomain myhostname = mx.example.com mynetworks = 127.0.0.0/8, [::1]/128, 192.168.128.0/24, [global:ip:6::]/64 newaliases_path = /usr/bin/newaliases.postfix non_smtpd_milters = inet:localhost:8891 proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix-2.10.1/README_FILES sample_directory = /usr/share/doc/postfix-2.10.1/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop smtpd_milters = inet:localhost:8891 smtpd_recipient_restrictions = check_policy_service inet:localhost:12350, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policyd-spf smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = yes smtpd_sasl_path = private/auth smtpd_sasl_type = dovecot smtpd_tls_CAfile = /etc/letsencrypt/live/example.com/chain.pem smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/letsencrypt/live/example.com/fullchain.pem smtpd_tls_dh1024_param_file = /etc/postfix/dhparam.pem smtpd_tls_key_file = /etc/letsencrypt/live/example.com/privkey.pem smtpd_tls_mandatory_ciphers = medium smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_security_level = may smtpd_use_tls = yes tls_medium_cipherlist = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 tls_preempt_cipherlist = no unknown_local_recipient_reject_code = 550 virtual_alias_domains = proxy:mysql:/etc/postfix/mysql-virtual_alias_domains.cf virtual_alias_maps = proxy:mysql:/etc/postfix/mysql-virtual_forwardings.cf, proxy:mysql:/etc/postfix/mysql-virtual_email2email.cf virtual_gid_maps = static:12 virtual_mailbox_base = /var/vmail virtual_mailbox_domains = proxy:mysql:/etc/postfix/mysql-virtual_domains.cf virtual_mailbox_maps = proxy:mysql:/etc/postfix/mysql-virtual_mailboxes.cf virtual_transport = dovecot virtual_uid_maps = static:89 postconf -Mf smtp inet n - n - - smtpd -o smtpd_recipient_restrictions= -o content_filter=spamassassin submission inet n - n - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_sasl_auth_enable=yes -o smtpd_reject_unlisted_recipient=no -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
Re: [CentOS] Postfix and virtual mail boxes.[SOLVED - kinda]
On 9/10/21 12:26 pm, Rob Kampen wrote: So, after many dozens of hours and sending test emails I have found a solution (work around) that appears to work okay. It is now different to the original two MX servers I cloned from, in that the maillog shows a different cycle of processing, and it now fails a truly unknown mailbox much later in the process - thus higher workload on my MX. But the key thing is that it does now do the virtual_alias checks on incoming emails on port 25 before rejecting. if your MX is not rejecting messages to invalid recipients right away but instead bounces the messages later on you become a backscatter source (See https://www.backscatterer.org/?target=bounces). your server needs a properly configured list of valid recipients so it knows right away what recipients to accept and which ones to reject. No idea why this third MX is behaving differently. It has a dual stack IP, so I disabled IPv6 access and tried again, but that certainly wasn't the cause of the difference in processing. If you can provide the output of the following two commands it would be very helpful in troubleshooting your problem: postconf -nf postconf -Mf Also of great help would relevant logs for one message that is giving you issues. These should be in /var/log/maillog and contain a connection line followed by a number of postfix/smtpd lines, please copy all the logs for *one* message. Please do not attempt to enable verbose logging (it only adds in a lot of extra unneeded info that detracts from finding the real problem) and it is unnecessary to provide log lines from non-postfix processes. It should be noted that the two initial MX machines have an extra line in the maillog that is the second logged step in the process, and goes something like: Oct 8 19:00:58 mx policyd-spf[16055]: prepend Received-SPF: None (mailfrom) identity=mailfrom; client-ip=209.85.210.180; helo=mail-pf1-f180.google.com; envelope-from=r...@example.com; receiver= This is likely unrelated to the issue but may point to another issue having to do with a possibly incorrect policyd setup. We can cross that bridge after we've fixed the primary issue though (one issue at a time). After that processing steps are identical. It's likely that there may be something else subtle in the logs that we can spot that you are not noticing. Peter ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Postfix and virtual mail boxes.[SOLVED - kinda]
On 7/10/21 6:18 pm, Rob Kampen wrote: Hi, not sure this is the best place to go for my problem, but hoping someone can point me to the correct or better place. I have two currently working CentOS 7 based email servers that host a number of virtual domains and users and delivers mail just fine - for correctly addressed vmail inboxes AND for alias addressed emails and domains. These all use postfix, dovecot, amavisd, clamAV, spamassassin, mysql (mariadb) and roundcubemail I use port 25 for the world email delivery - no auth needed as only accepts hosted virtual domain addressed email. and port 587 for user client MUA to send mail - smtp with STARTTLS auth needed port 993 for MUA IMAP access to account mail boxes A couple of weeks ago I rolled up a new minimal virtual server (also CentOS 7) and basically copied the same setup as the other two and have now spent far too long trying to get it going. My initial problem was that I set up mydestination to include $mydomain - this has the consequence of seeing any mail@$mydomain as local email and trying to deliver to the local machine rather than the vmail inboxes. So after setting up a new domain just for the MX we moved on. All three installations use mysql (mariadb) as the data store for domains, alias domains, user vmail accounts, and forwardings (virtual alias mapping) The mysql bits work just fine as I can send mail from an MUA client and they deliver correctly via an authenticated session on port 587 - i.e to any world email address AND to local vmail boxes, including those addressed via an alias and/or alias domain. What fails to work on the new installation, but works fine on my two legacy servers, is mail addressed via an alias. Specifically an alias domain. Even adding the complete alias email address to the forwardings table doesn't work. e.g. let's say we have an email domain '@example.com' and an alias domain '@example.org'. Needed so I can migrate my clients from one server to the other in a staged manner. Thus the new server is set up to operate as the MX for @example.org and @example.com but needs to alias redirect the incoming emails being sent to f...@example.org and deliver them to the vmail location for f...@example.com (i.e. we have no vmail locations for @example.org only for @example.com So if I send a test email via my MUA (using port 587 and hence authenticated) it does the alias lookups and translations needed and correctly delivers the mail. However if I send an email to f...@example.org from say a gmail account, it arrives at my new server and promptly gets bounced with a 550 5.7.1 error - no such email address. After doing diff on the main.cf and master.cf from all three servers the only differences are the myhostname, mynetworks (new one is dual stack IP4/IPv6 and thus includes [::1]/128), smtpd_tls certificate names, and the virtual_uid_maps - all expected and accounted for. master.cf are identical Many dozens of google searches and reading far too many pages, has left me with no idea why my new server doesn't accept alias directed emails via port 25. All the documentation indicates that alias lookups and translations are performed by postfix - all the time. receive_override_options is not set. So, after many dozens of hours and sending test emails I have found a solution (work around) that appears to work okay. It is now different to the original two MX servers I cloned from, in that the maillog shows a different cycle of processing, and it now fails a truly unknown mailbox much later in the process - thus higher workload on my MX. But the key thing is that it does now do the virtual_alias checks on incoming emails on port 25 before rejecting. No idea why this third MX is behaving differently. It has a dual stack IP, so I disabled IPv6 access and tried again, but that certainly wasn't the cause of the difference in processing. It should be noted that the two initial MX machines have an extra line in the maillog that is the second logged step in the process, and goes something like: Oct 8 19:00:58 mx policyd-spf[16055]: prepend Received-SPF: None (mailfrom) identity=mailfrom; client-ip=209.85.210.180; helo=mail-pf1-f180.google.com; envelope-from=r...@example.com; receiver= After that processing steps are identical. My work around makes the steps a little more different. So the work around is changing master.cf From # == # service type private unpriv chroot wakeup maxproc command + args # (yes) (yes) (yes) (never) (100) # == smtp inet n - n - - smtpd -o content_filter=spamassassin TO # == # service type private unpriv chroot wakeup maxproc command + args
