Re: Delay re-sending message following onward rejection
Dominic Raferd: > > You can reduce the number of MX hosts to try to just 1, by setting > > up an SMTP client for gmail etc. that has > > > > smtp -o smtp_mx_session_limit=1 > > > > With that, Postfix still tries multiple MX hosts until one responds, > > and you will have $min_backoff-time or more to dig into the queue > > and remove the offending message. > > Thanks Wietse that sounds like a good plan, how would I set up a > 'special' smtp client for gmail servers with the reduced mx session > limit? I think I know how to create the special client in master.cf, > but how do I tell the delivery agent (queue?) to use this smtp client > for gmail destinations (and not for others)? /etc/postfix/main.cf: transport_maps = hash:/etc/postfix/transport /etc/postfix/transport: gmail.com: one-mx-session ...ditto for other domains... /etc/postfix/master.cf: one-mx-sessionunix - - - - - smtp -o smtp_mx_session_limit=1 And leave smtp_mx_address_limit alone (don't set it to 1, otherwise mail will back up in the queue for no good reason). Wietse
Re: Delay re-sending message following onward rejection
On 4 January 2017 at 16:52, Wietse Venemawrote: > Dominic Raferd: >> My idea is to force a delay (2 seconds say) between the initial >> failure and the re-sending of the same email (same queue-id) to the >> secondary mx (or fallback relay) - in the intervening time the message >> may be pulled from the queue. Following earlier advice from Wietse >> here I am already using 'smtp_transport_rate_delay = 5s', but although >> helpful in other respects this does not delay the re-sending in this >> case. > > That won't work well because the 'next mx' is chosen in the SMTP > client (i.e. the message is not returned to the queue between > attempts to deliver the message to different MX hosts) and we don't > want to pause the SMTP client as that would prevent it from delivering > other email. > > (the destination rate delay is per destination and must therefore > be implemented in the queue manager). > > You can reduce the number of MX hosts to try to just 1, by setting > up an SMTP client for gmail etc. that has > > smtp -o smtp_mx_session_limit=1 > > With that, Postfix still tries multiple MX hosts until one responds, > and you will have $min_backoff-time or more to dig into the queue > and remove the offending message. > Thanks Wietse that sounds like a good plan, how would I set up a 'special' smtp client for gmail servers with the reduced mx session limit? I think I know how to create the special client in master.cf, but how do I tell the delivery agent (queue?) to use this smtp client for gmail destinations (and not for others)?
Re: multi-instances: how to discriminate "master" in process list
> On Jan 4, 2017, at 2:44 AM, Marco Pizzoliwrote: > > Hi all, > I have a multi-instance setup. > By doing "ps -ef", as expected, I see a lot of "master" processes. > Is there a way to see which master is related to which instance at a glance? "postfix status" will output the pid of each instance's master process. -- Viktor.
Re: Delay re-sending message following onward rejection
Dominic Raferd: > My idea is to force a delay (2 seconds say) between the initial > failure and the re-sending of the same email (same queue-id) to the > secondary mx (or fallback relay) - in the intervening time the message > may be pulled from the queue. Following earlier advice from Wietse > here I am already using 'smtp_transport_rate_delay = 5s', but although > helpful in other respects this does not delay the re-sending in this > case. That won't work well because the 'next mx' is chosen in the SMTP client (i.e. the message is not returned to the queue between attempts to deliver the message to different MX hosts) and we don't want to pause the SMTP client as that would prevent it from delivering other email. (the destination rate delay is per destination and must therefore be implemented in the queue manager). You can reduce the number of MX hosts to try to just 1, by setting up an SMTP client for gmail etc. that has smtp -o smtp_mx_session_limit=1 With that, Postfix still tries multiple MX hosts until one responds, and you will have $min_backoff-time or more to dig into the queue and remove the offending message. Wietse
Delay re-sending message following onward rejection
Is there a way to delay re-sending a message following an onward rejection? I am getting occasional messages back from an onward server (gmail) about a bad email; within a second we remove the bad email from the queue and block the originator's ip. But sometimes the first of these actions is too late and the mail has already been sent to a secondary mx, which then generates a 2nd rejection message (and so on). My idea is to force a delay (2 seconds say) between the initial failure and the re-sending of the same email (same queue-id) to the secondary mx (or fallback relay) - in the intervening time the message may be pulled from the queue. Following earlier advice from Wietse here I am already using 'smtp_transport_rate_delay = 5s', but although helpful in other respects this does not delay the re-sending in this case. If you wondered, such a message will not have originated from my side; it is crud from outside addressed to one of our users and my server is relaying it to one of our own gmail mailboxes. We have a raft of preventative measures including dnsbl lists and amavis, but bad stuff still gets through occasionally. Gmail's testing is better and for us definitive - if Gmail won't accept an email it is unacceptable by definition because we use Gmail mailboxes, so I just want rid of it. I've given example log messages below and added my comments. Thanks for any help. Dominic # we've received an incoming mail, processed it and sent it on to gmail - but it is rejected (the code and text message varies) 2017-01-04 16:20:07 mymx1 postfix/smtp[14166]: F064F3EC59: host gmail-smtp-in.l.google.com[74.125.195.27] said: 421-4.7.0 [164.132.228.222 15] Our system has detected that this message is 421-4.7.0 suspicious due to the nature of the content and/or the links within. 421-4.7.0 To best protect our users from spam, the message has been blocked. 421-4.7.0 Please visit 421 4.7.0 https://support.google.com/mail/answer/188131 for more information. p199si57699706wmd.1 - gsmtp (in reply to end of DATA command) # relay-enforcer spots the log entry above, intervenes and uses postsuper to remove the message from the queue... 2017-01-04 16:20:07 mymx1 postfix/postsuper[14358]: F064F3EC59: removed 2017-01-04 16:20:07 mymx1 postfix/postsuper[14358]: Deleted: 1 message 2017-01-04 16:20:07 mymx1 relay-enforcer-long: removed F064F3EC59 from postfix queues (if any) # ... and places a log message containing originator's ip - this will be picked up by matching fail2ban jail 2017-01-04 16:20:07 mymx1 relay-enforcer-long: bannable F064F3EC59 6C6893E96A posta.skfuzlethaz.hu 178.48.9.21 # action by fail2ban jail (extract from syslog) 2017-01-04 16:20:08 mymx1 fail2ban.actions[1580]: NOTICE [relay-enforcer-long] Ban 178.48.9.21 # but what's this? The same message had *already* (before the deletion, presumably) been sent out to an alternate gmail mx - and is rejected again 2017-01-04 16:20:08 mymx1 postfix/smtp[14166]: F064F3EC59: host alt1.gmail-smtp-in.l.google.com[64.233.163.27] said: 421-4.7.0 [164.132.228.222 15] Our system has detected that this message is 421-4.7.0 suspicious due to the nature of the content and/or the links within. 421-4.7.0 To best protect our users from spam, the message has been blocked. 421-4.7.0 Please visit 421 4.7.0 https://support.google.com/mail/answer/188131 for more information. a207si42877320lfd.65 - gsmtp (in reply to end of DATA command)
Re: multi-instances: how to discriminate "master" in process list
Marco Pizzoli: > Hi all, > I have a multi-instance setup. > By doing "ps -ef", as expected, I see a lot of "master" processes. > Is there a way to see which master is related to which instance at a glance? # postfix status postfix/postfix-script: the Postfix mail system is running: PID: 1290 postfix-m1/postfix-script: the Postfix mail system is running: PID: 1435 Wietse
Re: Make smtp client talk through SSH tunnel?
Matthias Andree: > Am 04.01.2017 um 12:47 schrieb Wietse Venema: > > > > You need to make smtp(8) talk to a TCP port (or UNIX-domain port), > > an arrange for a little daemon that listens on that port, and that > > invokes ssh when a connection is established to that port. Then > > the little daemon shuttles bits up and down. Such an on-demand > > TCP relay could be done in Perl, Python, or any capable language. > > Thanks. The TCP relay and asynchronous or triggered start (even through > autossh) is exactly what I'm considering brittle and am trying to avoid; > I am looking for something synchronous. Looks like we still don't have > that without someone extending the smtp client's source code. Sorry, I fail to see the problem with using a synchronous SMTP proxy service that manages the synchronous connection for you. Such a synchronous service could even synchronously respond with 4xx when a connection is unavailable. Wietse
RE: multi-instances: how to discriminate "master" in process list
If you are on linux, would you find these commands helpful ? pgrep master|xargs ps –fp Example Output: UIDPID PPID C STIME TTY TIME CMD root 7437 1 0 2016 ?00:29:41 /usr/libexec/postfix/master pgrep -u postfix|xargs ps –fp Example Output: UIDPID PPID C STIME TTY STAT TIME CMD postfix 7449 7437 0 2016 ?S 52:35 qmgr -l -t fifo -u postfix 7837 7437 0 2016 ?S 0:19 tlsmgr -l -t unix -u postfix 13859 7437 0 05:54 ?S 0:02 scache -l -t unix -u postfix 17144 7437 0 06:36 ?S 0:11 trivial-rewrite -n rewrite -t unix -u postfix 28545 7437 0 Jan03 ?S 0:02 anvil -l -t unix -u postfix 29491 7437 0 09:14 ?S 0:00 pickup -l -t fifo -u postfix 29620 7437 0 09:15 ?S 0:00 smtp -t unix -u postfix 29623 7437 0 09:15 ?S 0:00 smtp -t unix -u postfix 29624 7437 0 09:15 ?S 0:00 smtp -t unix -u postfix 29625 7437 0 09:16 ?S 0:00 smtp -t unix -u postfix 29626 7437 0 09:16 ?S 0:00 smtp -t unix -u postfix 29627 7437 0 09:16 ?S 0:00 smtp -t unix -u postfix 30435 7437 0 09:26 ?S 0:00 smtp -t unix -u postfix 30436 7437 0 09:26 ?S 0:00 smtp -t unix -u postfix 30437 7437 0 09:26 ?S 0:00 smtp -t unix -u postfix 30438 7437 0 09:26 ?S 0:00 smtp -t unix -u postfix 30439 7437 0 09:26 ?S 0:00 smtp -t unix -u postfix 30461 7437 0 09:26 ?S 0:00 smtp -t unix -u postfix 30896 7437 0 09:31 ?S 0:00 smtpd -n smtp -t inet -u -o stress= postfix 30910 7437 0 09:31 ?S 0:00 trivial-rewrite -n rewrite -t unix -u postfix 31302 7437 0 09:35 ?S 0:00 bounce -z -n defer -t unix -u postfix 31303 7437 0 09:35 ?S 0:00 bounce -z -n defer -t unix -u postfix 31304 7437 0 09:35 ?S 0:00 smtp -t unix -u postfix 31305 7437 0 09:35 ?S 0:00 bounce -z -n defer -t unix -u postfix 31306 7437 0 09:35 ?S 0:00 smtp -t unix -u postfix 31307 7437 0 09:35 ?S 0:00 smtp -t unix -u postfix 31323 7437 0 09:36 ?S 0:00 smtp -t unix -u postfix 31324 7437 0 09:36 ?S 0:00 smtp -t unix -u postfix 31325 7437 0 09:36 ?S 0:00 smtp -t unix -u postfix 31326 7437 0 09:36 ?S 0:00 smtp -t unix -u postfix 31340 7437 0 09:36 ?S 0:00 bounce -z -n defer -t unix -u postfix 31437 7437 0 09:37 ?S 0:00 cleanup -z -t unix -u postfix 31439 7437 0 09:37 ?S 0:00 cleanup -z -t unix -u postfix 31441 7437 0 09:37 ?S 0:00 smtpd -n smtp -t inet -u -o stress= postfix 31444 7437 0 09:37 ?S 0:00 smtpd -n smtp -t inet -u -o stress= postfix 31446 7437 0 09:37 ?S 0:00 smtpd -n smtp -t inet -u -o stress= postfix 31451 7437 0 09:37 ?S 0:00 cleanup -z -t unix -u -Angelo Fazzina Operating Systems Programmer / Analyst University of Connecticut, UITS, SSG, Server Systems 860-486-9075 From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Marco Pizzoli Sent: Wednesday, January 4, 2017 2:45 AM To: Postfix usersSubject: multi-instances: how to discriminate "master" in process list Hi all, I have a multi-instance setup. By doing "ps -ef", as expected, I see a lot of "master" processes. Is there a way to see which master is related to which instance at a glance? So that I can rapidly say this pid is for instance 1, and so on... What I am looking at is, ideally, a way to see the process name something like "master-instance1" i.e. rspamd does exactly this, assigning a different process name just to explicit the "incarnation" of that process Thank you in advance Marco
Re: Avoiding spam blacklists
On 2016-12-28 09:36, Alice Wonder wrote: On 12/28/2016 12:28 AM, John Fawcett wrote: On 12/28/2016 08:32 AM, Alice Wonder wrote: Virtual machine for a web application, it is still in testing. reverse DNS is properly set up. Postfix only listens on the local host. Linux firewall drops anything not to port 80, 443, or a custom high number port I use for SSH. This postfix is not an open relay, or a relay for anything on the Internet, it only exists so the web application can send e-mail. SPF for the domain is correctly set up, DKIM for the host is correctly set up, when it sends an e-mail and I inspect it - it passes the rDNS, SPF, and DKIM checks. So far it has only sent e-mails to addresses I control as the web application is still in testing. Yet yesterday the IP address ended up on Spamhaus blacklist. I am 100% confident that no one else was sending e-mail from that IP address, I'm a bit puzzled as to how the IP address got added to the blacklist, but I was told that Spamhaus sometimes just adds an entire subnet if more than one IP on the subnet was sending spam, and that's probably what happened. I think that is irresponsible of Spamhaus if that is what they are doing, but is there something more I can do other than correct rDNS, SPF, and DKIM to avoid getting on a blacklist? if you know which of the spamhaus lists it was you can check out its policy. Each list has its own specific criteria. Also if you were recently assigned the ip the listing may predate your activity. John The IP is relatively new to me, about two months, but it was not on the list before as I use Spamhaus on my other mail servers and mail from it was not being rejected until yesterday. I did go through the manual removal process and that worked, but I'm worried about it happening again. So what dis Spamhaus say? Why is your IP listed? Did you actually mass mail?
Re: Make smtp client talk through SSH tunnel?
Am 04.01.2017 um 12:47 schrieb Wietse Venema: > > You need to make smtp(8) talk to a TCP port (or UNIX-domain port), > an arrange for a little daemon that listens on that port, and that > invokes ssh when a connection is established to that port. Then > the little daemon shuttles bits up and down. Such an on-demand > TCP relay could be done in Perl, Python, or any capable language. Thanks. The TCP relay and asynchronous or triggered start (even through autossh) is exactly what I'm considering brittle and am trying to avoid; I am looking for something synchronous. Looks like we still don't have that without someone extending the smtp client's source code.
Re: [postfix-users] Make smtp client talk through SSH tunnel?
> > workaround is to establish SSH port forwarding asynchronously, and that > > is a fragile setup that I would like to replace by something synchronous > You need to make smtp(8) talk to a TCP port (or UNIX-domain port), > an arrange for a little daemon that listens on that port, and that > invokes ssh when a connection is established to that port. Then > the little daemon shuttles bits up and down. Such an on-demand > TCP relay could be done in Perl, Python, or any capable language. If stunnel is not an option I'd prefer autossh. It preopens the tunnel and restarts if needed. It is rock stable and it is off the shelf. Gabor
Re: Make smtp client talk through SSH tunnel?
Matthias Andreewrote: > Greetings and a happy new year, > > > I still am in a situation where I occasionally need to have an SMTP > client (preferable Postfix's) talk through an SSH tunnel. > > I know we have the smtp(8) client, and we have the pipe(8) client for > injecting RFC5322 stuff into commands, but what I need is some form of > the smtp(8) client talk to the ssh command (with certain arguments) > instead of establishing the TCP connection by itself. The current > workaround is to establish SSH port forwarding asynchronously, and that > is a fragile setup that I would like to replace by something synchronous > that doesn't need to set up TCP tunnels when I can instead have "ssh -W > host:port" that talks through stdin/stdout. > > I haven't seen such a feature in the 3.1 release notes - what needs to > happen that smtp can - perhaps via special syntax - be made to talk > through a command's stdio rather than through BSD sockets? Have you considered inetd/xinetd based solution? see "Running stunnel in inetd mode" at https://www.stunnel.org/howto.html It will allow you to execute "ssh forwarder" when connection is made to local ports of your choice. You may combine it with iptables based "transparent proxy". -- A. Filip
Re: Make smtp client talk through SSH tunnel?
On 01/04/2017 12:47 PM, Wietse Venema wrote: > Matthias Andree: >> Greetings and a happy new year, >> >> >> I still am in a situation where I occasionally need to have an SMTP >> client (preferable Postfix's) talk through an SSH tunnel. >> >> I know we have the smtp(8) client, and we have the pipe(8) client for >> injecting RFC5322 stuff into commands, but what I need is some form of >> the smtp(8) client talk to the ssh command (with certain arguments) >> instead of establishing the TCP connection by itself. The current >> workaround is to establish SSH port forwarding asynchronously, and that >> is a fragile setup that I would like to replace by something synchronous >> that doesn't need to set up TCP tunnels when I can instead have "ssh -W >> host:port" that talks through stdin/stdout. >> >> I haven't seen such a feature in the 3.1 release notes - what needs to >> happen that smtp can - perhaps via special syntax - be made to talk >> through a command's stdio rather than through BSD sockets? > You need to make smtp(8) talk to a TCP port (or UNIX-domain port), > an arrange for a little daemon that listens on that port, and that > invokes ssh when a connection is established to that port. Then > the little daemon shuttles bits up and down. Such an on-demand > TCP relay could be done in Perl, Python, or any capable language. > > Wietse I think that socat can do that. https://linux.die.net/man/1/socat John
Re: Make smtp client talk through SSH tunnel?
Matthias Andree: > Greetings and a happy new year, > > > I still am in a situation where I occasionally need to have an SMTP > client (preferable Postfix's) talk through an SSH tunnel. > > I know we have the smtp(8) client, and we have the pipe(8) client for > injecting RFC5322 stuff into commands, but what I need is some form of > the smtp(8) client talk to the ssh command (with certain arguments) > instead of establishing the TCP connection by itself. The current > workaround is to establish SSH port forwarding asynchronously, and that > is a fragile setup that I would like to replace by something synchronous > that doesn't need to set up TCP tunnels when I can instead have "ssh -W > host:port" that talks through stdin/stdout. > > I haven't seen such a feature in the 3.1 release notes - what needs to > happen that smtp can - perhaps via special syntax - be made to talk > through a command's stdio rather than through BSD sockets? You need to make smtp(8) talk to a TCP port (or UNIX-domain port), an arrange for a little daemon that listens on that port, and that invokes ssh when a connection is established to that port. Then the little daemon shuttles bits up and down. Such an on-demand TCP relay could be done in Perl, Python, or any capable language. Wietse
Make smtp client talk through SSH tunnel?
Greetings and a happy new year, I still am in a situation where I occasionally need to have an SMTP client (preferable Postfix's) talk through an SSH tunnel. I know we have the smtp(8) client, and we have the pipe(8) client for injecting RFC5322 stuff into commands, but what I need is some form of the smtp(8) client talk to the ssh command (with certain arguments) instead of establishing the TCP connection by itself. The current workaround is to establish SSH port forwarding asynchronously, and that is a fragile setup that I would like to replace by something synchronous that doesn't need to set up TCP tunnels when I can instead have "ssh -W host:port" that talks through stdin/stdout. I haven't seen such a feature in the 3.1 release notes - what needs to happen that smtp can - perhaps via special syntax - be made to talk through a command's stdio rather than through BSD sockets? Thanks. Best regards, Matthias
Re: Small question: how do I see in the log on which port the connection is made?
Is there a list somewhere of just what options can be logged with “-o syslog_name=x” ? > On 4 Jan 2017, at 04:12, Viktor Dukhovniwrote: > > >> On Jan 3, 2017, at 7:36 PM, Bradley Giesbrecht >> wrote: >> >> Add syslog_name to the appropriate service in master.cf. >> >> -o syslog_name=postfix-25 >> -o syslog_name=postfix-587 > > The recommended name format is postfix/detail not postfix-detail. > See the stock master.cf in Postfix 3.2: > > https://github.com/vdukhovni/postfix/blob/master/postfix/conf/master.cf > (lines 17,18): > >#submission inet n - n - - smtpd ># -o syslog_name=postfix/submission > > -- > -- > Viktor. >
Re: Rate-limiting access to postfix on the firewall, what are decent numbers (depending on overall traffic)?
But the point is OVH servers have no need to access submission, pop3, or imap. I have reduced the attack surface. I can receive email from OVH servers since I provide no filtering on port 25 other than a few RBLs. I don't condone filtering port 25. Leave that to the RBLs. But don't get in the RBLs sights by hanging out in a bad neighborhood. This list is supposed to support postfix and this conversation has strayed off that topic, granted partially my fault. My apologies. Original Message From: Dominic Raferd Sent: Wednesday, January 4, 2017 1:21 AM To: li...@lazygranch.com Cc: postfix-users@postfix.org Subject: Re: Rate-limiting access to postfix on the firewall, what are decent numbers (depending on overall traffic)? On 4 January 2017 at 08:53,wrote: > Reread. I don't not block port 25. > > I assure you, OVH has been used for C by hackers. Angler comes to mind. > > Original Message > From: Dominic Raferd > Sent: Tuesday, January 3, 2017 11:42 PM > To: postfix-users@postfix.org; li...@lazygranch.com > Subject: Re: Rate-limiting access to postfix on the firewall, what are decent > numbers (depending on overall traffic)? > > On 4 January 2017 at 02:16, < > li...@lazygranch.com> wrote: >> >> http://bgp.he.net/AS16276#_prefixes >> I'd switch to 587 and block everything OVH. Actually I do just that since >> OVH is on my Web Access blocking list, which I also use to block all mail >> ports other than 25. >> >> OVH VPS are often used by hackers. I think they are as low as $3 a month. > > > This is rash; we use OVH and we are not spammers - we need a static ip > (as it is not offered by our ISP) and they provide one at a good > price. You risk blocking genuine emails - this one included (except > you may receive it via the list mailserver). It's the false positive risk. 'Some [vps provider X] servers have been used by hackers' does not mean 'all connections from [vps provider X] servers are attempted hacks'. Of course you are entitled to your own decision.
Re: Rate-limiting access to postfix on the firewall, what are decent numbers (depending on overall traffic)?
On 4 January 2017 at 08:53,wrote: > Reread. I don't not block port 25. > > I assure you, OVH has been used for C by hackers. Angler comes to mind. > > Original Message > From: Dominic Raferd > Sent: Tuesday, January 3, 2017 11:42 PM > To: postfix-users@postfix.org; li...@lazygranch.com > Subject: Re: Rate-limiting access to postfix on the firewall, what are decent > numbers (depending on overall traffic)? > > On 4 January 2017 at 02:16, < > li...@lazygranch.com> wrote: >> >> http://bgp.he.net/AS16276#_prefixes >> I'd switch to 587 and block everything OVH. Actually I do just that since >> OVH is on my Web Access blocking list, which I also use to block all mail >> ports other than 25. >> >> OVH VPS are often used by hackers. I think they are as low as $3 a month. > > > This is rash; we use OVH and we are not spammers - we need a static ip > (as it is not offered by our ISP) and they provide one at a good > price. You risk blocking genuine emails - this one included (except > you may receive it via the list mailserver). It's the false positive risk. 'Some [vps provider X] servers have been used by hackers' does not mean 'all connections from [vps provider X] servers are attempted hacks'. Of course you are entitled to your own decision.
Re: Rate-limiting access to postfix on the firewall, what are decent numbers (depending on overall traffic)?
Reread. I don't not block port 25. I assure you, OVH has been used for C by hackers. Angler comes to mind. Original Message From: Dominic Raferd Sent: Tuesday, January 3, 2017 11:42 PM To: postfix-users@postfix.org; li...@lazygranch.com Subject: Re: Rate-limiting access to postfix on the firewall, what are decent numbers (depending on overall traffic)? On 4 January 2017 at 02:16, < li...@lazygranch.com> wrote: > > http://bgp.he.net/AS16276#_prefixes > I'd switch to 587 and block everything OVH. Actually I do just that since OVH > is on my Web Access blocking list, which I also use to block all mail ports > other than 25. > > OVH VPS are often used by hackers. I think they are as low as $3 a month. This is rash; we use OVH and we are not spammers - we need a static ip (as it is not offered by our ISP) and they provide one at a good price. You risk blocking genuine emails - this one included (except you may receive it via the list mailserver).