Spare your breath. I have solved my issue AWS-SES, and it behaves well now with Pigeonhole Sieve Vacation (s. patch attached)
Many thanks for all your thoughts. I will leave the list now. Best regards Rolf
pigeonhole-vacation-sender-fix.patch
Description: Binary data
> Am 11.02.2023 um 09:01 schrieb Paul Kudla <p...@scom.ca>: > > > Ok again just trying to help > > ___ > The question on why I use AWS-SES as my outbound mail relays can be simply > answered with the attribute „superior reputation“. > ___ > > > that being said, again an experience thing that most people do not know about > ! > > opensrs (i use them for my domain registration thus i had a wholesale account > setup and could interact with tech support on other issues, this being an > example of one.) > > that being said .... > > reputations are mostly purchased now a days, people do not block server's > based on reputation that in most cases is actually paid for. > > For example years ago I had a customer receive an email from a supplier in > china > > Suppliers MUST have a bank transfer etc before they will ship > > My customer lost 15000.00 us in a bogus transfer because opensrs's email > servers were on a spf whitelist? > > What can i say experience, spf is designed to prevent spam emails but more so > verify that they came from an authorized server. > > Believe it or not, the supplier got hacked, the hacker setup a duplicate > email with the same email address on an opensrs server. > > SPF would have caught it except opensrs's email server are whitelisted ! > > Customer lost the money, unable to recover and opensrs denied any > responsibilty for paying to be whitelisted. > > My SPF system is now patched to skip any whitelist via SPF as it functions as > it should now. > > Microsoft, Google etc are also other culprites on bypassing things in the > name of saving some bandwidth. > > Anything within there systems are generally automatically whitelisted, Again > another customer, they are on Outlook 365, I received an email that said our > domains were suspended etc, nothing new there get those all the time, the > worrisom part was someone setup an email server, then proxied through > microsoft in a way that was very clever, had an spf record and everything > setup, but they were using microsoft as a proxy to a microsoft account so the > mail got delivered when again it should have bounced back as invalid sender. > > I understand this is not directly related but reputations are paid for and > relays will never fully work upstream as it is dependant on what the upstream > provider changes from time to time > > Its a cat and mouse game that will never end. > > > Again just trying to help. > > > Happy Saturday !!! > Thanks - paul > > Paul Kudla > > > Scom.ca Internet Services <http://www.scom.ca> > 004-1009 Byron Street South > Whitby, Ontario - Canada > L1N 4S3 > > Toronto 416.642.7266 > Main 1.866.411.7266 > Fax 1.888.892.7266 > Email p...@scom.ca > > On 2/10/2023 9:27 AM, Dr. Rolf Jansen wrote: >> As stated elsewhere, the severe problem of incomprehensible OoO notice comes >> not because I relay MY outbound mails via Amazon’s SES but because some of >> MY PEERS (senders of the original messages and receivers of OoO notices) do >> or perhaps other relays which do funny manipulations of envelope sender and >> some headers in the message body as well. That said, my usage of AWS-SES may >> probably raise similar problems to the receivers of our mails wanting to >> return OoO notices to our users. >> The question on why I use AWS-SES as my outbound mail relays can be simply >> answered with the attribute „superior reputation“. My experience is that SES >> is blocked nowhere, except perhaps in North Korea, I didn’t try yet. For >> professional emails this is mission critical, and you cannot even get close >> to this if you setup somewhere, somehow your best practice own relay. >> This reputation has of course to do with SES controlling bounces. SES does >> control outgoing rate. SES does control the domain of the sender's address >> (envelop and From:) has been registered with the service. They do everything >> that SES ist not being compromised by any criminals. For me this is >> important, and then I need to live with the peculiarities and annoyances and >> perhaps find workarounds. >> Best regards >> Rolf >>> Am 10.02.2023 um 10:30 schrieb Paul Kudla <p...@scom.ca>: >>> >>> Good morning, >>> I have been following this post for a bit and would like to share >>> experience please and thanks. >>> >>> This is not meant to give a solution but save some massive frustration with >>> other system as i have gone through the same issues overall. >>> >>> In general I found found over the past few years all the big boys are >>> forcing all the private systems into standards that are not really defined >>> and get implemented willy nilly. >>> >>> Just because microsoft starts a standard, then google picks up on it then >>> AWS and then yahoo etc etc in any order does not mean its a proper approach. >>> >>> That being said is there any reason why you are not sending the emails >>> directly yourself, ie why are you using a proxy. >>> >>> I found (for example) when forwarding an email from @scom.ca to gmail for >>> example all the headers, dkim, spf records are all passed along which >>> resulted in emails never being allowed to be delivered. >>> >>> Although this may be your issue directly or indirectly what i found is to >>> forward to a gmail.com account i had to program the gmail.com account to >>> pop my server. This does work well but only for gmail.com >>> >>> I have other customers where i try to pop the email from whatever system >>> (which does work) but when i forward to an account on my system postfix >>> rewrite the header from address to the mailxx.scom.ca email server name >>> being used to forward the email which generates the same issues you are >>> having in the headers being rewritten not showing the from address? >>> >>> My server's are setup with custom python programming filters developed over >>> ten years and i can not seem to control anything either? >>> >>> I get you do production stuff (so do my customers) which is why it might be >>> better to send via a postfix instance that you are in control of >>> >>> of couse this does require a static ip etc which i dont know if you have >>> access to or not? >>> >>> but i think this would save a lot of frustration trying to be "COMPATIBLE" >>> with everyone else out there that do not even follow their own standards? >>> >>> >>> Just though i would pass this info along, trying to help ? >>> >>> >>> >>> Happy Friday !!! >>> Thanks - paul >>> >>> Paul Kudla >>> >>> >>> Scom.ca Internet Services <http://www.scom.ca> >>> 004-1009 Byron Street South >>> Whitby, Ontario - Canada >>> L1N 4S3 >>> >>> Toronto 416.642.7266 >>> Main 1.866.411.7266 >>> Fax 1.888.892.7266 >>> Email p...@scom.ca >>> >>> On 2/10/2023 7:18 AM, Dr. Rolf Jansen wrote: >>>>> Am 08.02.2023 um 20:03 schrieb Michael Peddemors <mich...@linuxmagic.com>: >>>>> >>>>> Dovecot vacation message issues.. >>>>> Tough for any system to do correctly. >>>>> >>>>>> The problem here is that inbound mails from third parties utilizing >>>>>> AWS-SES come in with an unpersonalized envelope address and SES takes >>>>>> returns to this as bounce messages and changes the body's From: to >>>>>> „mailer-dae...@xx-zzzz-1.amazonses.com“, which is not even our >>>>>> MAILER-DAEMON but the one of the receiver of our reply. So the receiver >>>>>> gets no chance to know from the headers the identity of whom replied - >>>>>> he may assume it from the context the actual message, though. >>>>> >>>>> We addressed this by NOT returning vacation messages to systems that >>>>> don't use 'proper' values in the MAIL FROM.. Eg Mailing Lists, Sender >>>>> Rewrite schemes, and a slurry of other rules. >>>> Who is we? Your organization or the Pigeonhole developers? Actually, the >>>> question is, whether this is addressed somewhere in Pigeonhole’s code >>>> already? >>>>> But the problem is that if you are using the header From, or Reply-To >>>>> etc, it's too easy to be sending to forged email addresses. >>>>> >>>>> Vacation bombing attacks for instance.. >>>> You got a point here, and of course I want to prevent this. >>>>> Now, there are legitimate cases of the MAIL FROM and header from not >>>>> aligning, so it is best to send to the MAIL FROM addresses.. IF you don't >>>>> send it to certain MAIL FROM formats, usually by not responding to >>>>> anything with mailing list identifiers, auto-suppress headers, and a few >>>>> others, you only end up with clean MAIL FROM to respond to. >>>> From the point of the view of our industrial customers, who are operating >>>> processes with our chemicals, this consideration is irrelevant. If they >>>> inform a production issue by mail to the responsible service technician, >>>> they expect an immediate response, since a production stop is >>>> unacceptable. OoO notices play a role here, because we would inform >>>> alternative addresses and fone numbers for attending the support case. >>>> That said, with Pigeonhole, we are almost there. >>>>> But if you have an example that is particularly bothering you, and >>>>> represents your problem, we can walk through that as an example. >>>> I send an email from an account of a mail server (Postfix/Dovecot - >>>> outbound relay SES) running on an AWS-EC2 instance in São Paulo (Brazil) >>>> to another mail address of mine of a mail server (Postfix/Dovecot direct >>>> MX) on an AWS-EC2 instance in Frankfurt Germany, and here the Pigeonhole’s >>>> vacation reply is activated. >>>> In the following I changed my real mail address in Brazil to >>>> r...@example.br and the real one in Germany to r...@example.de: >>>> The Point of view of the both involved Postfixes of said transactions are: >>>> Sender (Brazil): >>>> postfix/submission/smtpd[29165]: 97006638E8: >>>> client=201-68-62-42.dsl.telesp.net.br[201.68.62.42], sasl_method=CRAM-MD5, >>>> sasl_username=r...@example.br >>>> postfix/cleanup[29182]: 97006638E8: >>>> message-id=<707a1777-8f09-4335-99ba-70c0367c1...@example.br> >>>> postfix/qmgr[2058]: 97006638E8: from=<r...@example.br>, size=39626, >>>> nrcpt=1 (queue active) >>>> postfix/smtp[29183]: 97006638E8: to=<r...@example.de>, >>>> relay=email-smtp.sa-east-1.amazonaws.com[52.67.192.29]:587, delay=0.37, >>>> delays=0.05/0.01/0.13/0.18, dsn=2.0.0, status=sent (250 Ok >>>> 010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000000) >>>> Receiver (Germany): >>>> postfix/smtpd[86956]: connect from >>>> d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] >>>> postfix/smtpd[86956]: A44AB676E3: >>>> client=d215-2.smtp-out.sa-east-1.amazonses.com[23.249.215.2] >>>> postfix/cleanup[86957]: A44AB676E3: >>>> message-id=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000...@sa-east-1.amazonses.com> >>>> postfix/qmgr[915]: A44AB676E3: >>>> from=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000...@sa-east-1.amazonses.com>, >>>> size=40862, nrcpt=1 (queue active) >>>> postfix/local[86958]: A44AB676E3: passing <r...@example.de> to >>>> transport=lmtp >>>> postfix/lmtp[86959]: A44AB676E3: to=<r...@example.de>, >>>> relay=mail.example.de[private/dovecot-lmtp], delay=0.94, >>>> delays=0.83/0.01/0.04/0.07, dsn=2.0.0, status=sent (250 2.0.0 >>>> <r...@example.de> +/goFGgl5mOwUwEAEYr/fg Saved) >>>> Sender of the OoO (Germany): >>>> postfix/qmgr[915]: 60242676F0: from=<r...@example.de>, size=1177, nrcpt=1 >>>> (queue active) >>>> postfix/cleanup[86957]: 60242676F0: >>>> message-id=<dovecot-sieve-1676027240-34335...@example.de> >>>> postfix/pickup[62932]: 60242676F0: uid=xxxx from=<r...@example.de> >>>> postfix/smtp[86963]: 60242676F0: >>>> to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000...@sa-east-1.amazonses.com>, >>>> relay=email-smtp.eu-central-1.amazonaws.com[3.74.180.161]:587, >>>> delay=0.31, delays=0.01/0.02/0.13/0.15, dsn=2.0.0, status=sent (250 Ok >>>> 010701863b022070-bb3e4541-8306-4438-b649-8879ec8ff666-000000) >>>> Receiver of the OoO notice (Brazil): >>>> postfix/smtpd[29184]: connect from >>>> d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] >>>> postfix/smtpd[29184]: 5F1346394E: >>>> client=d215-244.smtp-out.sa-east-1.amazonses.com[23.249.215.244] >>>> postfix/cleanup[29182]: 5F1346394E: >>>> message-id=<010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000...@sa-east-1.amazonses.com> >>>> postfix/qmgr[2058]: 5F1346394E: from=<>, size=3150, nrcpt=1 (queue active) >>>> postfix/local[29185]: 5F1346394E: passing <r...@example.br> to >>>> transport=lmtp >>>> postfix/lmtp[29186]: 5F1346394E: to=<r...@example.br>, >>>> relay=mail.example.br[private/dovecot-lmtp], delay=0.05, >>>> delays=0.01/0.01/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 >>>> <r...@example.br> RRZHGWwl5mMDcgAAjZNhtw Saved) >>>> The headers of the received OoO message (I removed the DKIM stuff for >>>> clarity) are: >>>> Return-Path: <> >>>> Delivered-To: r...@example.br >>>> Received: from mail.example.br >>>> by example.br with LMTP >>>> id RRZHGWwl5mMDcgAAjZNhtw >>>> (envelope-from <>) >>>> for <r...@example.br>; Fri, 10 Feb 2023 08:07:24 -0300 >>>> Received: from d215-244.smtp-out.sa-east-1.amazonses.com >>>> (d215-244.smtp-out.sa-east-1.amazonses.com [23.249.215.244]) >>>> (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) >>>> (No client certificate requested) >>>> by mail.example.br (Postfix) with ESMTPS id 5F1346394E >>>> for <r...@example.br>; Fri, 10 Feb 2023 08:07:24 -0300 (-03) >>>> DKIM-Signature: ... >>>> X-Sieve: Pigeonhole Sieve 0.5.20 (149edcf2) >>>> Message-ID: >>>> <010301863b022c93-9842f45a-85ec-419f-8199-00e027888394-000...@sa-east-1.amazonses.com> >>>> Date: Fri, 10 Feb 2023 11:07:23 +0000 >>>> From: mailer-dae...@sa-east-1.amazonses.com >>>> To: r...@example.br >>>> Subject: Out of Office - automatic notice >>>> In-Reply-To: >>>> <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000...@sa-east-1.amazonses.com> >>>> References: >>>> <010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000...@sa-east-1.amazonses.com> >>>> Auto-Submitted: auto-replied (vacation) >>>> Precedence: bulk >>>> X-Auto-Response-Suppress: All >>>> MIME-Version: 1.0 >>>> Content-Type: text/plain; charset=utf-8 >>>> Content-Transfer-Encoding: 8bit >>>> Feedback-ID: >>>> 1.sa-east-1.E7DnIghDRHBvwjrx2QL73gyWAj+NYISxiq7bqOVD27E=:AmazonSES >>>> X-SES-Outgoing: 2023.02.10-23.249.215.244 >>>> Note how there is not a single reference of the real origin of the OoO >>>> message, r...@example.de. Instead we see From: >>>> mailer-dae...@sa-east-1.amazonses.com. The receiver of this message need >>>> to guess the real address from the outside context. >>>> The actual problem is that Pigeonhole responds to >>>> to=<010301863b0211fe-9416f5b2-7e18-4c03-a5e5-2204dd7946f8-000...@sa-east-1.amazonses.com> >>>> instead of to=<r...@example.de>. Amazon SES of the original sender (not >>>> our relay) takes this as a bounce and changes the relevant headers. This >>>> is not under our control. >>>> Perhaps this could be mitigated by adding a Reply-To: header which informs >>>> the original sender. Then the receiver of the OoO noice would at least >>>> have a reference to what and whom this is about. >>>> Best regards >>>> Rolf >>>>> On 2023-02-08 13:21, Dr. Rolf Jansen wrote: >>>>>> Am 08.02.2023 um 12:57 schrieb Michael Peddemors >>>>>> <mich...@linuxmagic.com>: >>>>>>> On 2023-02-08 07:47, Dr. Rolf Jansen wrote: >>>>>>>> Am 08.02.2023 um 12:23 schrieb Michael Peddemors >>>>>>>> <mich...@linuxmagic.com>: >>>>>>>>> On 2023-02-07 14:57, Dr. Rolf Jansen wrote: >>>>>>>>>> Am 07.02.2023 um 19:49 schrieb Michael Peddemors >>>>>>>>>> <mich...@linuxmagic.com>: >>>>>>>>>>> On 2023-02-07 13:33, jeremy ardley wrote: >>>>>>>>>>>> On 8/2/23 05:08, Dr. Rolf Jansen wrote: >>>>>>>>>>>>> Am 07.02.2023 um 17:54 schrieb jeremy ardley<jer...@ardley.org>: >>>>>>>>>>>>>> On 7/2/23 22:01, Dr. Rolf Jansen wrote: >>>>>>>>>>>>>> To begin with, usage of Amazons Simple Email Service (SES) is >>>>>>>>>>>>>> mandatory for outgoing mails from AWS-EC2 instances. >>>>>>>>>>>>>> I run AWS-EC2 instances using postfix to send a receive mail. >>>>>>>>>>>>>> They can send direct assuming I set up suitable SPF, but they >>>>>>>>>>>>>> typically forward mail to another host under my control that is >>>>>>>>>>>>>> not on AWS to use as the outgoing server. >>>>>>>>>>>>> OK, that’s another use case. Many do use a full fledged >>>>>>>>>>>>> Postfix/Dovecot installation. However the outgoing port 25 into >>>>>>>>>>>>> the internet is blocked by AWS, and therefore we may either use a >>>>>>>>>>>>> third party relay for our outgoing emails or may use SES, which >>>>>>>>>>>>> is not that bad - except some unusual peculiarities. >>>>>>>>>>>>> >>>>>>>>>>>> This is off topic, but to be precise: >>>>>>>>>>>> - AWS throttles but does not block traffic to a *destination* port >>>>>>>>>>>> 25. >>>>>>>>>>>> - The *origin* port on the EC2 instance is an unprivilged port, >>>>>>>>>>>> not port 25 >>>>>>>>>>>> - If you use a relayhost you typically send from an unprivilged >>>>>>>>>>>> EC2 port to port 587 on the relay host >>>>>>>>>>>> Jeremy >>>>>>>>>>> >>>>>>>>>>> And if you DO intend to send out to port 25, remember to update the >>>>>>>>>>> PTR on your EC2 instance. >>>>>>>>>> I respond off-list. Generally a good hint but it’s like trying to >>>>>>>>>> bath salts (https://en.wikipedia.org/wiki/Bath_salts) when you are >>>>>>>>>> taking a shower. >>>>>>>>>> https://aws.amazon.com/premiumsupport/knowledge-center/ec2-port-25-throttle/ >>>>>>>>>> Although the link text (ec2-port-25-throttle) suggests throttling, >>>>>>>>>> it is actually blocking - read the text. This cannot be removed from >>>>>>>>>> standard EC2 instances. >>>>>>>>>> This also corresponds to my experience. On the console of an AWS-EC2 >>>>>>>>>> instance, I entered just now: >>>>>>>>>> telnet cyclaero.com 25 >>>>>>>>>> This hangs until timeout. At the same time cyclaero.com received >>>>>>>>>> emails from the internet on port 25. >>>>>>>>>> Trying 3.126.110.101... >>>>>>>>>> telnet: connect to address 3.126.110.101: Operation timed out >>>>>>>>>> telnet: Unable to connect to remote host >>>>>>>>> >>>>>>>>> There are a lot of malicous or compromised EC2 instances..It is good >>>>>>>>> they block port by default, and the one day delay before unblocking >>>>>>>>> (you can request it) helps stop those short term spam instances. >>>>>>>>> >>>>>>>>> Now, wish they blocked port 587 by default ;) >>>>>>>>> >>>>>>>>> Or at least tracked the attacks. >>>>>>>>> >>>>>>>>> Strange though, that people still try to set up email on EC2, doesn't >>>>>>>>> make sense, and it isn't the cheapest alternative.
