Re: Postfix SMTPUTF8 support (unicode email addresses)
* Wietse Venema [2014-07-15 19:33]: > Proudly presenting Postfix SMTPUTF8 support! Below is text from > the RELEASE_NOTES file for postfix-2.12-20140715, to be uploaded > later today. Aaand Google has announced that it will support this for GMail: http://googleblog.blogspot.com/2014/08/a-first-step-toward-more-global-email.html So I expect there might be an increase in interest for this. Again Postfix is at the bleeding edge, nicely done. :) Regards Sebastian -- GPG Key: 0x93A0B9CE (F4F6 B1A3 866B 26E9 450A 9D82 58A2 D94A 93A0 B9CE) 'Are you Death?' ... IT'S THE SCYTHE, ISN'T IT? PEOPLE ALWAYS NOTICE THE SCYTHE. -- Terry Pratchett, The Fifth Elephant
RE: How to fetch From address from header via Postfix head_check?
Viktor,
>>This rather severely limits the usability of your MSA. It cannot support
>>ordinary email sent to multiple recipients or Bcc'ed. Also you say this is
>>an MSA, and >>yet claim the mail is sent by external senders outside OSU.
>>How are these two statements compatible? Is this an MSA processing outbound
>>mail generated >>internally at OSU, or simply an outbound relay, forwarding
>>mail whose recipients are external to your email systems (possibly your users
>>hosted outside).
>>Explain your system more clearly.
Main email system is Microsoft exchange system. The Exchange Hub servers
deliver the all outbound mails (internal users send emails to external users
or external users send emails to internal users BUT whose email addresses are
forwarding to his/her external mailboxes) to Postfix servers. The postfix
servers receive all emails which the recipient addresses are external email
addresses. So I think it simply an outbound relay, forwarding mail whose
recipients are external to your email systems.
>>Mail you've accepted (whether inbound or outbound) that is then forwarded to
>>Microsoft for a hosted mailbox SHOULD NOT be spam filtered by Microsoft.
That resposibility falls on your systems as the original systems that
>>receive the mail from the external sender.
Currently the situation is all outbound emails are sent to MICROSOFT antispam
system - EOP for scanning before they are delivered to destination external
mailboxes. Sometimes internal users' mailboxes are possibly compromised to be
abused to send a lot of outbound junks.
>>The systems you use to forward mail to Microsoft for your own hosted users,
>>MUST be whitelisted by Microsoft for delivery to the hosted users in
>>question, >>with NO spam filters applied by them.
The fact is the systems we currently use are not whitelisted by Microsoft for
delivery to the hosted users in question with NO spam filters applied by them.
As I say above - Sometimes internal users' mailboxes are possibly compromised
to be abused to send a lot of outbound junks.
>>If Microsoft cannot do this for you, find a better email hosting provider.
>>You're wasting time attacking the wrong problem.
The decision will be made by higher level of managements, not me. Sometimes the
effort used to attack the wrong problem is not fairly wasting time.
Thanks,
Carl
-Original Message-
From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org]
On Behalf Of Viktor Dukhovni
Sent: Tuesday, August 05, 2014 5:46 PM
To: postfix-users@postfix.org
Subject: Re: How to fetch From address from header via Postfix head_check?
On Tue, Aug 05, 2014 at 09:28:24PM +, Xie, Wei wrote:
> > What you're proposing is not viable, and seems to serve no purpose.
> > You should explain the problem you're trying to solve by adding
> > these, rather than the problems you're having doing so.
>
> When the message hits our outbound Postfix servers, on an MSA the "To:"
> address only list one recipient. We do not need consider multiple
> recipients.
This rather severely limits the usability of your MSA. It cannot support
ordinary email sent to multiple recipients or Bcc'ed. Also you say this is an
MSA, and yet claim the mail is sent by external senders outside OSU. How are
these two statements compatible? Is this an MSA processing outbound mail
generated internally at OSU, or simply an outbound relay, forwarding mail whose
recipients are external to your email systems (possibly your users hosted
outside).
Explain your system more clearly.
> The problem is the nexthop - Microsoft antispam system due to their
> bugs is eating some outbound emails from non-osu.edu or
> non-ohio-state.edu senders to forwarding accounts. But their system
> does not eat the emails which are "Resent-From" from mailbox users
> ("Resent-From:" is appropriate when a user takes a message delivered
> to his mailbox (possibly long after initial delivery) and resends it
> to another user (typically not an original recipient). Our exchange engineers
> ask whether Postfix can add "Resent-From:
> " for emails to forwarding accounts like mailbox
> accounts resent the emails to bypass Microsoft antispam system (this
> is one of all kinds attempts).
Mail you've accepted (whether inbound or outbound) that is then forwarded to
Microsoft for a hosted mailbox SHOULD NOT be spam filtered by Microsoft. That
resposibility falls on your systems as the original systems that receive the
mail from the external sender.
The systems you use to forward mail to Microsoft for your own hosted users,
MUST be whitelisted by Microsoft for delivery to the hosted users in question,
with NO spam filters applied by them.
If Microsoft cannot do this for you, find a better email hosting provider.
You're wasting time attacking the wrong problem.
--
Viktor.
Re: How to fetch From address from header via Postfix head_check?
Am 06.08.2014 um 14:02 schrieb Xie, Wei:
> Viktor,
>
>>> This rather severely limits the usability of your MSA. It cannot support
>>> ordinary email sent to multiple recipients or Bcc'ed. Also you say this is
>>> an MSA, and >>yet claim the mail is sent by external senders outside OSU.
>>> How are these two statements compatible? Is this an MSA processing
>>> outbound mail generated >>internally at OSU, or simply an outbound relay,
>>> forwarding mail whose recipients are external to your email systems
>>> (possibly your users hosted outside).
>>> Explain your system more clearly.
>
> Main email system is Microsoft exchange system. The Exchange Hub servers
> deliver the all outbound mails (internal users send emails to external users
> or external users send emails to internal users BUT whose email addresses are
> forwarding to his/her external mailboxes) to Postfix servers. The postfix
> servers receive all emails which the recipient addresses are external email
> addresses. So I think it simply an outbound relay, forwarding mail whose
> recipients are external to your email systems.
by the way traditional smtp outside forward may break any time, by
strict spf,dmarc,dkim, perhaps workaround with "outlook forward rules
only" may work
>
>>> Mail you've accepted (whether inbound or outbound) that is then forwarded
>>> to Microsoft for a hosted mailbox SHOULD NOT be spam filtered by Microsoft.
>>> >>That resposibility falls on your systems as the original systems that
>>> receive the mail from the external sender.
>
> Currently the situation is all outbound emails are sent to MICROSOFT
> antispam system - EOP for scanning before they are delivered to destination
> external mailboxes. Sometimes internal users' mailboxes are possibly
> compromised to be abused to send a lot of outbound junks.
Ok so far, whats the problem ?
>
>>> The systems you use to forward mail to Microsoft for your own hosted users,
>>> MUST be whitelisted by Microsoft for delivery to the hosted users in
>>> question, >>with NO spam filters applied by them.
>
> The fact is the systems we currently use are not whitelisted by Microsoft for
> delivery to the hosted users in question with NO spam filters applied by
> them. As I say above - Sometimes internal users' mailboxes are possibly
> compromised to be abused to send a lot of outbound junks.
as Viktor wrote, that sounds like "design problem" with no direct
relation to postfix
>
>>> If Microsoft cannot do this for you, find a better email hosting provider.
>>> You're wasting time attacking the wrong problem.
>
> The decision will be made by higher level of managements, not me. Sometimes
> the effort used to attack the wrong problem is not fairly wasting time.
however decision was made ,it does not change tec facts, re-think your
smtp design, i.e let exchange deliver out itself, use other antispam
practice etc
>
>
> Thanks,
>
> Carl
>
> -Original Message-
> From: owner-postfix-us...@postfix.org
> [mailto:owner-postfix-us...@postfix.org] On Behalf Of Viktor Dukhovni
> Sent: Tuesday, August 05, 2014 5:46 PM
> To: postfix-users@postfix.org
> Subject: Re: How to fetch From address from header via Postfix head_check?
>
> On Tue, Aug 05, 2014 at 09:28:24PM +, Xie, Wei wrote:
>
>>> What you're proposing is not viable, and seems to serve no purpose.
>>> You should explain the problem you're trying to solve by adding
>>> these, rather than the problems you're having doing so.
>>
>> When the message hits our outbound Postfix servers, on an MSA the "To:"
>> address only list one recipient. We do not need consider multiple
>> recipients.
>
> This rather severely limits the usability of your MSA. It cannot support
> ordinary email sent to multiple recipients or Bcc'ed. Also you say this is
> an MSA, and yet claim the mail is sent by external senders outside OSU. How
> are these two statements compatible? Is this an MSA processing outbound mail
> generated internally at OSU, or simply an outbound relay, forwarding mail
> whose recipients are external to your email systems (possibly your users
> hosted outside).
>
> Explain your system more clearly.
>
>> The problem is the nexthop - Microsoft antispam system due to their
>> bugs is eating some outbound emails from non-osu.edu or
>> non-ohio-state.edu senders to forwarding accounts. But their system
>> does not eat the emails which are "Resent-From" from mailbox users
>> ("Resent-From:" is appropriate when a user takes a message delivered
>> to his mailbox (possibly long after initial delivery) and resends it
>> to another user (typically not an original recipient). Our exchange
>> engineers ask whether Postfix can add "Resent-From:
>> " for emails to forwarding accounts like mailbox
>> accounts resent the emails to bypass Microsoft antispam system (this
>> is one of all kinds attempts).
>
> Mail you've accepted (whether inbound or outbound) that is then forwa
RE: How to fetch From address from header via Postfix head_check?
Robert,
>> by the way traditional smtp outside forward may break any time, by strict
>> spf,dmarc,dkim, perhaps workaround with "outlook forward rules only" may
>> work
You may be right. But so far the delivery is still working except for the
problem coming from Microsoft antispam system - EOP.
>> Ok so far, whats the problem ?
The problem is next HOP - Microsoft antispam system EOP due to their bugs is
eating some outbound emails from non-osu.edu or non-ohio-state.edu senders to
forwarding accounts. But their system does not eat the emails which are
"Resent-From" from mailbox users ("Resent-From:" is appropriate when a user
takes a message delivered to his mailbox (possibly long after initial delivery)
and resends it to another user (typically not an original recipient). Our
exchange engineers ask whether Postfix can add "Resent-From: " for emails to forwarding accounts like mailbox accounts resent the
emails to bypass the problem of Microsoft antispam system (this is one of all
kinds attempts).
>> as Viktor wrote, that sounds like "design problem" with no direct relation
>> to postfix
Currently there is no use to discuss "design problem", which is not what I
would like to talk about it. Only want to know whether Postfix add
"Resent-From: " for emails to forwarding accounts like
mailbox accounts resent the emails to bypass the problem Microsoft antispam
system (this is one of all kinds attempts) due to their bugs.
>> however decision was made ,it does not change tec facts, re-think your smtp
>> design, i.e let exchange deliver out itself, use other antispam practice etc
All outbound mails have to be passed by Security scanning for sensitive data,
then Postfix delivers outbound emails out to Microsoft antispam system for
spam/virus scanning This design is the result which many departments work
together based on current business requirement and technical requirement. Any
changes are not easy to such a large system.
Thanks for your time!!!
Carl
-Original Message-
From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org]
On Behalf Of Robert Schetterer
Sent: Wednesday, August 06, 2014 8:38 AM
To: postfix-users@postfix.org
Subject: Re: How to fetch From address from header via Postfix head_check?
Am 06.08.2014 um 14:02 schrieb Xie, Wei:
> Viktor,
>
>>> This rather severely limits the usability of your MSA. It cannot support
>>> ordinary email sent to multiple recipients or Bcc'ed. Also you say this is
>>> an MSA, and >>yet claim the mail is sent by external senders outside OSU.
>>> How are these two statements compatible? Is this an MSA processing
>>> outbound mail generated >>internally at OSU, or simply an outbound relay,
>>> forwarding mail whose recipients are external to your email systems
>>> (possibly your users hosted outside).
>>> Explain your system more clearly.
>
> Main email system is Microsoft exchange system. The Exchange Hub servers
> deliver the all outbound mails (internal users send emails to external users
> or external users send emails to internal users BUT whose email addresses are
> forwarding to his/her external mailboxes) to Postfix servers. The postfix
> servers receive all emails which the recipient addresses are external email
> addresses. So I think it simply an outbound relay, forwarding mail whose
> recipients are external to your email systems.
by the way traditional smtp outside forward may break any time, by strict
spf,dmarc,dkim, perhaps workaround with "outlook forward rules only" may
work
>
>>> Mail you've accepted (whether inbound or outbound) that is then forwarded
>>> to Microsoft for a hosted mailbox SHOULD NOT be spam filtered by Microsoft.
>>> >>That resposibility falls on your systems as the original systems that
>>> receive the mail from the external sender.
>
> Currently the situation is all outbound emails are sent to MICROSOFT
> antispam system - EOP for scanning before they are delivered to destination
> external mailboxes. Sometimes internal users' mailboxes are possibly
> compromised to be abused to send a lot of outbound junks.
Ok so far, whats the problem ?
>
>>> The systems you use to forward mail to Microsoft for your own hosted users,
>>> MUST be whitelisted by Microsoft for delivery to the hosted users in
>>> question, >>with NO spam filters applied by them.
>
> The fact is the systems we currently use are not whitelisted by Microsoft for
> delivery to the hosted users in question with NO spam filters applied by
> them. As I say above - Sometimes internal users' mailboxes are possibly
> compromised to be abused to send a lot of outbound junks.
as Viktor wrote, that sounds like "design problem" with no direct relation to
postfix
>
>>> If Microsoft cannot do this for you, find a better email hosting provider.
>>> You're wasting time attacking the wrong problem.
>
> The decision will be made by higher level of managements, not me. S
Re: How to fetch From address from header via Postfix head_check?
On Wed, Aug 06, 2014 at 12:02:30PM +, Xie, Wei wrote: > Main email system is Microsoft exchange system. The Exchange Hub servers > deliver the all outbound mails (internal users send emails to external users > or external users send emails to internal users BUT whose email addresses are > forwarding to his/her external mailboxes) to Postfix servers. The postfix > servers receive all emails which the recipient addresses are external email > addresses. So I think it simply an outbound relay, forwarding mail whose > recipients are external to your email systems. Thus your claim that the mail will have a single recipient in the "To:" address, or will not employ "Bcc" is simply wrong. Your Postfix systems are outbound relays, not MSAs. Your real problem is that you're using the same servers to deliver *all* outbound email, both email that originates outside and needs to be forwarded for one of your externally hosted users, and email that your internally hosted users send out. This design severely limits your choices. When I did a related design for a previous employer with another hosting provider, I used dedicated systems to route just the mail for externally hosted users, separate from the outbound relays handling other mail. These dedicated systems were whitelisted by the provider, but restricted to delivery of mail to just the users in question, not the world at large. You're likely running into Sender-ID/SPF issues, where Microsoft applies anti-spoofing policy to your outbound machines, because you don't present a clean stream of email for just the hosted users. You need a more sophisticated design and a willingness from the hosting provider to work with you. You may need to hire an experienced consultant to help with the design and implementation, but the difficulty will be in assessing the skill of the consultant, this is hard to do, unless you're sufficiently skilled yourself. At the very least you may need to implement the SRS rewriting mechanism for forwarding mail in the age of SPF. You'll have to find a good Postfix SRS tutorial. This will likely resolve most of the problem you are reporting. -- Viktor.
RE: How to fetch From address from header via Postfix head_check?
Vicktor, >>Thus your claim that the mail will have a single recipient in the "To:" >>address, or will not employ "Bcc" is simply wrong. Your Postfix systems are >>outbound >>relays, not MSAs. Thanks for your correction. Learned the concept difference between MSA and outbound relay. >>Your real problem is that you're using the same servers to deliver >>*all* outbound email, both email that originates outside and needs to be >>forwarded for one of your externally hosted users, and email that your >>internally >>hosted users send out. >>This design severely limits your choices. When I did a related design for a >>previous employer with another hosting provider, I used dedicated systems to route just the mail for externally hosted users, separate from the outbound >>relays handling other mail. You are totally correct. Our previous email system was running like yours - forwarding accounts' mail flow to externally hosted users is separated from real outbound mail flow that our internally hosted users send out. Current Microsoft exchange system can't archive this goal. As you know, when politics involves business and technical decision, many reasonable things in the past have to be changed. >> These dedicated systems were whitelisted by the provider, but restricted to >> delivery of mail to just the users in question, not the world at large. Yes. Our previous system did so. >>You're likely running into Sender-ID/SPF issues, where Microsoft applies >>anti-spoofing policy to your outbound machines, because you don't present a >>clean >>stream of email for just the hosted users. So far it is OK. Microsoft antispam system EOP knows all IPs of our outbound servers. >>You need a more sophisticated design and a willingness from the hosting >>provider to work with you. You may need to hire an experienced consultant to help with the design and implementation, but the difficulty will be in >>assessing the skill of the consultant, this is hard to do, unless you're >>sufficiently skilled >>yourself. Thanks for your suggestion!!! When the design of current system was in assessing phase, one Microsoft experienced consultant pointed out our design problem, BUT ... >>At the very least you may need to implement the SRS rewriting mechanism for >>forwarding mail in the age of SPF. You'll have to find a good Postfix SRS tutorial. This will likely resolve most of the problem you are reporting. Thanks for your information about Postfix SRS tutorial. I will learn it. Thanks a lot! Carl -Original Message- From: owner-postfix-us...@postfix.org [mailto:owner-postfix-us...@postfix.org] On Behalf Of Viktor Dukhovni Sent: Wednesday, August 06, 2014 9:56 AM To: postfix-users@postfix.org Subject: Re: How to fetch From address from header via Postfix head_check? On Wed, Aug 06, 2014 at 12:02:30PM +, Xie, Wei wrote: > Main email system is Microsoft exchange system. The Exchange Hub servers > deliver the all outbound mails (internal users send emails to external users > or external users send emails to internal users BUT whose email addresses are > forwarding to his/her external mailboxes) to Postfix servers. The postfix > servers receive all emails which the recipient addresses are external email > addresses. So I think it simply an outbound relay, forwarding mail whose > recipients are external to your email systems. Thus your claim that the mail will have a single recipient in the "To:" address, or will not employ "Bcc" is simply wrong. Your Postfix systems are outbound relays, not MSAs. Your real problem is that you're using the same servers to deliver *all* outbound email, both email that originates outside and needs to be forwarded for one of your externally hosted users, and email that your internally hosted users send out. This design severely limits your choices. When I did a related design for a previous employer with another hosting provider, I used dedicated systems to route just the mail for externally hosted users, separate from the outbound relays handling other mail. These dedicated systems were whitelisted by the provider, but restricted to delivery of mail to just the users in question, not the world at large. You're likely running into Sender-ID/SPF issues, where Microsoft applies anti-spoofing policy to your outbound machines, because you don't present a clean stream of email for just the hosted users. You need a more sophisticated design and a willingness from the hosting provider to work with you. You may need to hire an experienced consultant to help with the design and implementation, but the difficulty will be in assessing the skill of the consultant, this is hard to do, unless you're sufficiently skilled yourself. At the very least you may need to implement the SRS rewriting mechanism for forwarding mail in the age of SPF. You'll have to find a good Postfix SRS tutorial. This will likely resolve most of the
Re: How to fetch From address from header via Postfix head_check?
On Wed, Aug 06, 2014 at 03:00:43PM +, Xie, Wei wrote: > You are totally correct. Our previous email system was running > like yours - forwarding accounts' mail flow to externally hosted > users is separated from real outbound mail flow that our internally > hosted users send out. Current Microsoft exchange system can't > archive this goal. As you know, when politics involves business > and technical decision, many reasonable things in the past have to > be changed. This is simply not the case. Exchange supports per-domain "connectors", that can send mail for selected domains to dedicated relays. In my design Exchange forwarded hosted recipients (a matter of populating the forwarding addresses that way) to a sythentic internal domain: user@.example.com where "example.com" was the primary internal domain. This domain was routed to dedicated Postfix outbound relays, which in turn performed rewriting of these addresses to the actual mailbox address at the provider. There are many ways to skin this cat. -- Viktor.
suggestion / log improvent
Hello,
the last day I had to search messages in our "poor man's second
chance" storage.
( an always_bcc solution ). *finding* messages was painful.
using my logging I could follow any message by its queueid. But
finally messages
are delivered by a local transport telling 10 times: yes, I saved
this message to a maildir.
I ask the dovecot-users list but found a simpler solution yet.
(http://www.dovecot.org/list/dovecot/2014-August/097369.html)
This is the current log of the postfix local delivery agent (which I
use on that specific host):
Aug 6 19:02:48 mailer postfix/local[6543]: 3hSzfc0Tv4z59xm:
to=, relay=local, delay=0.05,
delays=0.03/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to maildir)
with the attached patch I have queueid and messagefile combined in one line.
Aug 6 19:10:40 mailer postfix/local[30116]: 3hSzqh5c7Qz59xt:
to=, relay=local, delay=0.08,
delays=0.06/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to maildir,
/home/user/testmaildir/new/1407345040.Vfe00I15813dM811042.mailer)
that solve my current problem. Maybe other find it useful too.
Andreas
Index: postfix-2.11.1/src/local/maildir.c
===
--- postfix-2.11.1.orig/src/local/maildir.c 2012-01-25 01:41:08.0 +0100
+++ postfix-2.11.1/src/local/maildir.c 2014-08-06 19:18:10.0 +0200
@@ -242,7 +242,7 @@
defer_append : bounce_append)
(BOUNCE_FLAGS(state.request), BOUNCE_ATTR(state.msg_attr));
} else {
- dsb_simple(why, "2.0.0", "delivered to maildir");
+ dsb_simple(why, "2.0.0", "delivered to maildir: %s", newfile);
deliver_status = sent(BOUNCE_FLAGS(state.request),
SENT_ATTR(state.msg_attr));
}
