Winmail.dat - was:Re: [Mimedefang] Bare returns in message body
[EMAIL PROTECTED] wrote on 11/18/2005 08:53:30 PM: > straying a little OT but.. > ..i've been saying that a lot lately, with regard to Outlook sending > attachments as winmail.dat despite all my attempts to coerce it to do > otherwise. They complain their recipients can't open the attachments > but won't change their mail client so it's all my fault and i'm just > buck passing and not doing my job in letting the problem remain... :-( I've run into the dreaded winmail.dat files on the recieiving end (we don't use Exchange/Outlook) and every time I get the question, I explain that it is the sender's configuration, and include a link to a web site that explains the problem and why it is the sender that has to fix it and how they can do so. If it is a major honcho and they need the included information ASAP (don't they always?), I may dig out a winmail.dat decoder and extract the contents for them. Then I uninstall the decoder. I don't get the question often, every couple years or so. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
straying a little OT but.. On 19/11/2005, at 4:43 AM, Aleksandar Milivojevic wrote: Luckily, this time, you would be able to tell "I can't do anything about it, it is erorr in client's software that generated those emails, and it can only be fixed in that software". ..i've been saying that a lot lately, with regard to Outlook sending attachments as winmail.dat despite all my attempts to coerce it to do otherwise. They complain their recipients can't open the attachments but won't change their mail client so it's all my fault and i'm just buck passing and not doing my job in letting the problem remain... :-( (if anyone happens to have actually solved this i'd be more than happy to hear from you off-list...) ..S. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Quoting Steffen Kaiser <[EMAIL PROTECTED]>:
On Thu, 17 Nov 2005, Aleksandar Milivojevic wrote:
If any of $SuspisiousCharsIn* are true, I'm doing (as one of the
first things in
filter_begin, even before checking for viruses):
action_quarantine_entire_message('descriptive msg');
return action_bounce('descriptive msg');
I did so for some time, too, but had to disable it, because some
(important) people are subscribed to some CVS-has-changed
notification lists, which send embedded CRs and NULs. The sender was
complaining, that I'm the only person who thinks the mails are bad.
Sounds familiar. People are too often completely ignorant. They don't care
that simple upgrade of any component of email system (from email client
to SMTP
server to IMAP/POP3 server) can couse problems again (crashing the
email clients
or simply causing delivery problems again). The only thing they care about is
to delegate problem to somebody else. If your organization decides to swtich
to Cyrus IMAPD (for example) in the future, his emails are going to start
bouncing again. Luckily, this time, you would be able to tell "I can't do
anything about it, it is erorr in client's software that generated those
emails, and it can only be fixed in that software".
This message was sent using IMP, the Internet Messaging Program.
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Thu, 17 Nov 2005, Aleksandar Milivojevic wrote:
If any of $SuspisiousCharsIn* are true, I'm doing (as one of the first things
in
filter_begin, even before checking for viruses):
action_quarantine_entire_message('descriptive msg');
return action_bounce('descriptive msg');
I did so for some time, too, but had to disable it, because some
(important) people are subscribed to some CVS-has-changed notification
lists, which send embedded CRs and NULs. The sender was complaining, that
I'm the only person who thinks the mails are bad. (Well, I would probably
react this way as well, if I'd get only one reply.) I bet that they have
some newline problem (Mac vs. Unix vs. Windows), because these are huge
projects they are working within, so someone probably checks in the files
inconsistently.
Since then I bounce SuspiciousCharsInHeader only and treat them as
"infected by malware". No complains since then.
Bye,
--
Steffen Kaiser
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Quoting Tomasz Ostrowski <[EMAIL PROTECTED]>:
So I'd propose something like:
/* after message_contains_virus() */
if ($SuspiciousCharsInBody) {
action_rebuild();
}
If any of $SuspisiousCharsIn* are true, I'm doing (as one of the first
things in
filter_begin, even before checking for viruses):
action_quarantine_entire_message('descriptive msg');
return action_bounce('descriptive msg');
I have this setup for very long time, and so far zero complaints from
users. Even if there were complaints, this is part of anti-virus and
anti-spam policy,
so I couldn't do anything about it ;-)
Looking at the log files, more than 99% of bounced stuff are viruses and spam,
and remainder is mainly chain letters and similar stuff that nobody really
cares if it gets bounced. I've just checked this week's log files.
Almost all
bounced messages (due to suspisious chars in either body or headers) were from
senders like "[EMAIL PROTECTED]" (guess what those are). Only
two were from something that "looked" like it might have been real email
address. Checking the quarantine showed those two were viruses.
There was only one email adress in log files that was constantly
bounced because
of this (in the beggining, when we started using MIMEDefang), but it seems
whoever owned it have fixed his/hers email setup very fast after emails
started
to bounce. So bouncing isn't as bad as it may sound, it helps people to fix
problems ;-)
This message was sent using IMP, the Internet Messaging Program.
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Bare returns in message body
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On > Behalf Of Jan Pieter Cornet > Sent: woensdag 16 november 2005 10:56 > To: mimedefang@lists.roaringpenguin.com > Subject: Re: [Mimedefang] Bare returns in message body > Patching sendmail to reject on bare LF terminated lines is likely > asking for a LOT of trouble. Since traditionally sendmail doesn't care > if you used CRLF or just LF, it's likely that lots of (local, unix- > specific) programs submit messages using only LF line endings. Some > programs might even implicitly rely on the fact that sendmail > "corrects" the line endings. Indeed. I can tell you that, text-processing wise, almost everything I do on my server, in Perl scripts, starts with something that performs a "tr//" like function to convert CRLF to single LF. And in Perl you really kinda want to, as well; otherwise regex-es that count on $ and such (where you do *not* expect to have the end of your line to be a CR; or things like /bla\n/, etc.) will surely break. I have always found it a particular strength that MIME::Tools is not fussy about either CRLF, or just LF. And I thank sendmail for the same reason. If this were not the case, you'd have to keep the exact CRLF/LF sequences of each text, which really can yield quite unexpected, undesired results. And I'm happy both do their own converting to whatever format they see fit. Hence, lets not break things, and keep them the way they are (even if it does mean that there are scripts out there that do things with CR/LF that, strictly speaking, are not RFC compliant). - Mark System Administrator Asarian-host.org --- "If you were supposed to understand it, we wouldn't call it code." - FedEx ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Thu, 10 Nov 2005, David F. Skoll wrote:
> - There is no way to see a lone LF from milter.
Seems that it's no problem, because this should be a case also for
local mailer on unices. At least procmail saves files with bare
.
Does anybody use sendmail on MacOSX (unix to be or not unix to be) or
Windows to check it there?
> - There IS a way to see a lone CR.
So I'd propose something like:
/* after message_contains_virus() */
if ($SuspiciousCharsInBody) {
action_rebuild();
}
But then we should recheck rebuilt message for viruses - in case the
virus program has problems with bare . I don't know how to do
this (message_contains_virus() on modified message). Of course we
don't need to recheck attachments of this message (we build it so
we're sure there won't be anything unexpected) - only the message as
a whole.
Pozdrawiam
Tometzky
--
Best of prhn - najzabawniejsze teksty polskiego UseNet-u
http://prhn.dnsalias.org/
Chaos zawsze pokonuje porządek, gdyż jest lepiej zorganizowany.
[ Terry Pratchett ]
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Quoting Jan Pieter Cornet <[EMAIL PROTECTED]>: Patching sendmail to reject on bare LF terminated lines is likely asking for a LOT of trouble. Since traditionally sendmail doesn't care if you used CRLF or just LF, it's likely that lots of (local, unix- specific) programs submit messages using only LF line endings. Some programs might even implicitly rely on the fact that sendmail "corrects" the line endings. You are right there. Sendmail is one of the "be liberal what you accept, be strict what you send" projects. Unfortunately in this case, it is not strict enough in what it is sending... Kinda getting of topic for this mailing list... This message was sent using IMP, the Internet Messaging Program. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Tue, Nov 15, 2005 at 10:56:49AM -0600, Aleksandar Milivojevic wrote:
> >- There is no way to see a lone LF from milter.
> >- There IS a way to see a lone CR.
> >- There is no way to know if the CRLF you see in your milter was REALLY
> > a CRLF on the wire, or just a LF on the wire.
> >- When you send a body BACK to Sendmail, it makes no difference if you
> > terminate lines with LF or CRLF.
> >
> >What a mess.
>
> In other words, it should be Sendmail that should be patched to reject
> bare CR and bare LF. Patch against Sendmail (as config option)?
I have already experimented with rejecting on bare CR characters in
september 2004, and reported about it on this list. It does reject some
otherwise legal email.
Patching sendmail to reject on bare LF terminated lines is likely
asking for a LOT of trouble. Since traditionally sendmail doesn't care
if you used CRLF or just LF, it's likely that lots of (local, unix-
specific) programs submit messages using only LF line endings. Some
programs might even implicitly rely on the fact that sendmail
"corrects" the line endings.
So you could rephrase this as:
- sendmail treats both LF and CRLF identical, as legal line endings.
- a lone CR is treated like a regular character by sendmail, and
simply passed on (just like a NUL character, in fact...)
That latter part is breaking RFCs if you send the message on, so
MIMEDefang gives you the option of blocking it by flagging the message
with $SuspiciousCharsInBody.
IMHO, MIMEDefang should just mimick sendmail's behaviour of treating a
lone CR as a regular character, and pass it on. (Still flagging it as
$SuspiciousChar*, of course).
--
#!perl -wpl # mmfppfmpmmpp mmpffm <[EMAIL PROTECTED]>
$p=3-2*/[^\W\dmpf_]/i;s.[a-z]{$p}.vec($f=join('',$p-1?chr(sub{$_[0]*9+$_[1]*3+
$_[2]}->(map{/p|f/i+/f/i}split//,$&)+97):qw(m p f)[map{((ord$&)%32-1)/$_%3}(9,
3,1)]),5,1)='`'lt$&;$f.eig;# Jan-Pieter Cornet
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Aleksandar Milivojevic wrote: > IMO, MIMEDefang should at least set suspicios char flag when it > encounters bare CR or LF (well, the later is probably not possible > to detect on Unix systems). It does set that flag for a bare CR. Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Quoting "David F. Skoll" <[EMAIL PROTECTED]>: David F. Skoll wrote: Sigh. When you send body chunks back to Sendmail, it converts CRLF to LF, because it's writing it to a queue file, which is stored with UNIX-convention line endings. Also, when Sendmail reads the queue file and sends it to MIMEDefang, it converts LF to CRLF. So: - There is no way to see a lone LF from milter. - There IS a way to see a lone CR. - There is no way to know if the CRLF you see in your milter was REALLY a CRLF on the wire, or just a LF on the wire. - When you send a body BACK to Sendmail, it makes no difference if you terminate lines with LF or CRLF. What a mess. In other words, it should be Sendmail that should be patched to reject bare CR and bare LF. Patch against Sendmail (as config option)? I know of at least one popular IMAP server, namely Cyrus IMAPD, that does such checks and rejects messages that contain bare newlines. Furthermore, it is not conditional (unless you hack Cyrus code, it is not possible to turn it off). So this is nothing new, and could be considered common practice at many sites (be them aware of it or not). IMO, MIMEDefang should at least set suspicios char flag when it encounters bare CR or LF (well, the later is probably not possible to detect on Unix systems). This message was sent using IMP, the Internet Messaging Program. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
I'll say it first. Aleksandar is the man! > Choose a letter for beta options, and do them like '-B fix-line-endings=off' > (replace '-B' with whatever letter is free). Then you just need some short > portable code to parse key-value pair. Or you could have a letter for long > options '-o fix-line-endings=off'. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Quoting "David F. Skoll" <[EMAIL PROTECTED]>: [EMAIL PROTECTED] wrote: How about --dont-fix-line-endings Then I have to use getopt_long and portability goes to hell. :-( Choose a letter for beta options, and do them like '-B fix-line-endings=off' (replace '-B' with whatever letter is free). Then you just need some short portable code to parse key-value pair. Or you could have a letter for long options '-o fix-line-endings=off'. This message was sent using IMP, the Internet Messaging Program. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
David F. Skoll wrote: > The bigger question: Do we want to convert bare LF and bare CR to > (the equivalent of) CRLF? That is, treat CR, LF and CRLF as UNIX > line endings, so MIMEDefang sees them as LF, and when (if) we modify > the message bodies, we convert all the LFs to CRLF? Sigh. When you send body chunks back to Sendmail, it converts CRLF to LF, because it's writing it to a queue file, which is stored with UNIX-convention line endings. Also, when Sendmail reads the queue file and sends it to MIMEDefang, it converts LF to CRLF. So: - There is no way to see a lone LF from milter. - There IS a way to see a lone CR. - There is no way to know if the CRLF you see in your milter was REALLY a CRLF on the wire, or just a LF on the wire. - When you send a body BACK to Sendmail, it makes no difference if you terminate lines with LF or CRLF. What a mess. Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Wed, 09 Nov 2005, Jan Pieter Cornet wrote: > However, you're ALSO removing lone CRs in the process, CR characters > that a MUA will see, and might react upon (it might even trigger > a bug in the MUA... a bug which is scanned for in some virus scanner, > but that fails to detect it because the CR characters aren't there. > This is speculation, however). I remember a post to bugtraq that dealt with this as a security problem - I cannot google it though right now. There is a client software that treated bare and bare like but an antivirus gateway did not and haven't found an included virus. In that post there were 2 possible solutions: 1. reject bare and bare on the wire - not acceptable because of crappy SMTP software; 2. modify a message at gateway converting all bare 's and bare 's to , so we're sure that every software will treat this in the same way - this violates RFC (modifies a message at gateway) but it's not a problem with a message that already violates RFC. It would be nice for mimedefang to follow this second approach - every message violating should be converted before checking attachment names, using virus scanners or spamassassin and should be returned to sendmail also converted. Regards Tometzky -- ...although Eating Honey was a very good thing to do, there was a moment just before you began to eat it which was better than when you were... Winnie the Pooh ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Tomasz Ostrowski wrote: > It would be nice for mimedefang to follow this second approach - > every message violating should be converted before checking > attachment names, using virus scanners or spamassassin and should be > returned to sendmail also converted. You can ensure that Sendmail and client software sees exactly what MIMEDefang sees by calling action_rebuild(). The bigger question: Do we want to convert bare LF and bare CR to (the equivalent of) CRLF? That is, treat CR, LF and CRLF as UNIX line endings, so MIMEDefang sees them as LF, and when (if) we modify the message bodies, we convert all the LFs to CRLF? -- David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Wed, 9 Nov 2005, Jan Pieter Cornet wrote: Here's (imo) the fundamental problem here: the mimedefang filter is not given the same message that end user sees... lone CR characters will be removed from it (line ending CRLF will also be converted by sendmail or the local delivery agent to just LF, that's not the point). You can rebuilt the message when SuspiciousCharsInBody is true, then the message you filter in MIMEDefang is the same as the message the client sees. This is a good thing anyway, probably, because MIMETools reacts on ill-formatted MIME mails differently than a MUA possibly reacts. So you cannot be sure that both implementations sees the message the same. Bye, -- Steffen Kaiser ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Wed, Nov 09, 2005 at 05:02:43PM -0500, David F. Skoll wrote:
> I'm becoming convinced that I should leave the lone CRs in...
>
> Here's another question: What do we do about lone LF's? The problem
> here is that on UNIX, a lone LF looks just like a line termination.
> On a Mac, it looks like an embedded LF, and on Windows, I have no
> idea. So once again, the UNIX-based software might interpret the
> message differently from Windows- or Mac-based software.
The main issue is probably: what is the mail going to look like
when presented to the user, on a given system?
Since this is mimedefang, I think we can forget about OS9 macs.
On OSX, I'm fairly sure MIME::Tools generates LF-terminated lines,
but I'm too lazy to fire up my OSX machine to try it out.
Windows... does mimedefang even run on windows? And sendmail?
And milter? If it does, I bet the same crlf stripping is used
inside sendmail, leaving lone LFs just the same as CRLF line
endings, just like on unix.
Lone LFs are likely a non-issue, since they are treated
exacly the same as CRLF line endings in sendmail... but I'm only certain
about that on unix. (In fact, I'm currently receiving mails with lone
LFs... a quick look at the wire for about a minute did reveal one such
email... spam, though, but that doesn't mean much in a sampling of 1)
> What I propose doing is releasing a beta version of MIMEDefang that
> has a command-line option not to suppress lone CR's or LF's (though
> I'm rapidly running out of letters! :-))
unicode to the rescue! :)
> Existing behaviour of stripping CR's will continue by default. After
> enough people have tested that not stripping them doesn't blow things
> up, I will probably make it so we keep lone CR's and LF's intact.
That's probably the best way to handle this...
--
#!perl -wpl # mmfppfmpmmpp mmpffm <[EMAIL PROTECTED]>
$p=3-2*/[^\W\dmpf_]/i;s.[a-z]{$p}.vec($f=join('',$p-1?chr(sub{$_[0]*9+$_[1]*3+
$_[2]}->(map{/p|f/i+/f/i}split//,$&)+97):qw(m p f)[map{((ord$&)%32-1)/$_%3}(9,
3,1)]),5,1)='`'lt$&;$f.eig;# Jan-Pieter Cornet
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
[EMAIL PROTECTED] wrote: > How about --dont-fix-line-endings Then I have to use getopt_long and portability goes to hell. :-( Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Bare returns in message body
David F. Skoll wrote: > What I propose doing is releasing a beta version of MIMEDefang that > has a command-line option not to suppress lone CR's or LF's (though > I'm rapidly running out of letters! :-)) How about --dont-fix-line-endings -- Matthew.van.Eerde (at) hbinc.com 805.964.4554 x902 Hispanic Business Inc./HireDiversity.com Software Engineer ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Jan Pieter Cornet wrote: > However, you're ALSO removing lone CRs in the process, CR characters > that a MUA will see, and might react upon (it might even trigger > a bug in the MUA... a bug which is scanned for in some virus scanner, > but that fails to detect it because the CR characters aren't there. > This is speculation, however). I'm becoming convinced that I should leave the lone CRs in... Here's another question: What do we do about lone LF's? The problem here is that on UNIX, a lone LF looks just like a line termination. On a Mac, it looks like an embedded LF, and on Windows, I have no idea. So once again, the UNIX-based software might interpret the message differently from Windows- or Mac-based software. What I propose doing is releasing a beta version of MIMEDefang that has a command-line option not to suppress lone CR's or LF's (though I'm rapidly running out of letters! :-)) Existing behaviour of stripping CR's will continue by default. After enough people have tested that not stripping them doesn't blow things up, I will probably make it so we keep lone CR's and LF's intact. Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Wed, Nov 09, 2005 at 08:37:54AM -0500, David F. Skoll wrote:
> Jan Pieter Cornet wrote:
>
> > Nothing that I can think of easily. Perl has no problem with character
> > data, in general.
>
> Nevertheless, I seem to recall that many perl *modules* do have
> problems with embedded CR's. I believe (old versions of?) SpamAssassin
> had difficulties. And I suspect that MIME::Tools won't handle it well.
MIME::Tools (at least 5.418) has no problems with embedded CRs in the
body parts itself.
> Also, when I write the message body to INPUTMSG, I transform CRLF to
> LF so the line endings follow UNIX semantics. Otherwise, lots of
> Perl code *does* have problems.
Absolutely. That's not what I meant to suggest. You should do the
normal CRLF -> LF substition to transform "on the wire" format to
regular "unix line ending" format.
However, you're ALSO removing lone CRs in the process, CR characters
that a MUA will see, and might react upon (it might even trigger
a bug in the MUA... a bug which is scanned for in some virus scanner,
but that fails to detect it because the CR characters aren't there.
This is speculation, however).
Here's (imo) the fundamental problem here: the mimedefang filter is not
given the same message that end user sees... lone CR characters will be
removed from it (line ending CRLF will also be converted by sendmail
or the local delivery agent to just LF, that's not the point).
On Wed, Nov 09, 2005 at 01:56:46PM -0500, David F. Skoll wrote:
> I know of no good way to handle this. I'm inclined to simply reject
> messages with embedded CR's because MTAs sending such messages over
> the wire are violating a MUST NOT requirement.
In an ideal world, I'd love to enforce strict RFC compliance to all
incoming mail. It would remove a lot of spam. Unfortunately, there's
also a company out there, quite popular among users I might add,
that not only regularly violates RFCs, but seems to be making a sport
out of setting them on fire and sending a stampede of bad programmers
over it.
I mean... just look at how successful rfc-ignorant.org is...
If you care about the mail your users actually want to receive, some
RFCs have to be bent. Unfortunately.
--
#!perl -wpl # mmfppfmpmmpp mmpffm <[EMAIL PROTECTED]>
$p=3-2*/[^\W\dmpf_]/i;s.[a-z]{$p}.vec($f=join('',$p-1?chr(sub{$_[0]*9+$_[1]*3+
$_[2]}->(map{/p|f/i+/f/i}split//,$&)+97):qw(m p f)[map{((ord$&)%32-1)/$_%3}(9,
3,1)]),5,1)='`'lt$&;$f.eig;# Jan-Pieter Cornet
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
"David F. Skoll" <[EMAIL PROTECTED]> wrote: I know of no good way to handle this. I'm inclined to simply reject messages with embedded CR's because MTAs sending such messages over the wire are violating a MUST NOT requirement. I'm trying to diagnose MUAs that violate this. Mutt from one user, and Netscape 7 from another. Pretty strange. The Netscape user is sending Word attachments in French, and the Mutt user was sending a zip file. I'm at a loss!! Joe Brennan ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Les Mikesell wrote: > The RFC's deal only in 'wire level' representation during the SMTP > conversation. Right; that's what MIMEDefang gets handed by Sendmail. > Mail transports/delivery agents normally convert > to/from the local line endings. I don't think there is any such > thing as a universal file representation (the MIME folks really > missed a need here). Since MimeDefang's copy is sort-of still > in transport but is actually a local file copy it is bound to be > wrong for something. MIMEDefang's copy isn't really "in transport" unless you modify the message body. If you only alter headers, then the "in transport" copy is whatever's in Sendmail's queue file. I know of no good way to handle this. I'm inclined to simply reject messages with embedded CR's because MTAs sending such messages over the wire are violating a MUST NOT requirement. Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Wed, 2005-11-09 at 07:35, David F. Skoll wrote: > Joseph Brennan wrote: > > > We're interested in patching mimedefang.c to do this, but have this > > funny feeling that there is a good reason David removes them. What > > would it break? > > I can't remember why I remove them (it's been a while), but I did > have a good reason. > > RFC 2821 says: > >In addition, the appearance of "bare" "CR" or "LF" characters in text >(i.e., either without the other) has a long history of causing >problems in mail implementations and applications that use the mail >system as a tool. SMTP client implementations MUST NOT transmit >these characters except when they are intended as line terminators >and then MUST, as indicated above, transmit them only as a >sequence. > > RFC 2822 says: > >- CR and LF MUST only occur together as CRLF; they MUST NOT appear > independently in the body. > > Note that these are MUST NOT requirements, not SHOULD NOT requirements. > So in theory, it is OK to reject such messages. The RFC's deal only in 'wire level' representation during the SMTP conversation. Mail transports/delivery agents normally convert to/from the local line endings. I don't think there is any such thing as a universal file representation (the MIME folks really missed a need here). Since MimeDefang's copy is sort-of still in transport but is actually a local file copy it is bound to be wrong for something. -- Les Mikesell [EMAIL PROTECTED] ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Jan Pieter Cornet wrote: > Nothing that I can think of easily. Perl has no problem with character > data, in general. Nevertheless, I seem to recall that many perl *modules* do have problems with embedded CR's. I believe (old versions of?) SpamAssassin had difficulties. And I suspect that MIME::Tools won't handle it well. Also, when I write the message body to INPUTMSG, I transform CRLF to LF so the line endings follow UNIX semantics. Otherwise, lots of Perl code *does* have problems. Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
Joseph Brennan wrote: > We're interested in patching mimedefang.c to do this, but have this > funny feeling that there is a good reason David removes them. What > would it break? I can't remember why I remove them (it's been a while), but I did have a good reason. RFC 2821 says: In addition, the appearance of "bare" "CR" or "LF" characters in text (i.e., either without the other) has a long history of causing problems in mail implementations and applications that use the mail system as a tool. SMTP client implementations MUST NOT transmit these characters except when they are intended as line terminators and then MUST, as indicated above, transmit them only as a sequence. RFC 2822 says: - CR and LF MUST only occur together as CRLF; they MUST NOT appear independently in the body. Note that these are MUST NOT requirements, not SHOULD NOT requirements. So in theory, it is OK to reject such messages. Regards, David. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Bare returns in message body
On Tue, Nov 08, 2005 at 08:09:12PM -0500, Joseph Brennan wrote:
> So, why does Mimedefang remove the return characters as it creates
> INPUTMSG? We'd really like to know where they are so we can track
> down bugs in our own users' mail software.
Likely an oversight... but I'm just guessing. Normally, CR are removed
because the input stream has line endings encoded as CRLF while a normal
unix text file has lines ending in just LF... since the input is processed
byte-by-byte, if you find a "CR", you ignore it and set a flag "previous
was CR", then on the next character, if the flag is set, and the character
isn't a LF, it was a lone "CR" character.
> We're interested in patching mimedefang.c to do this, but have this
> funny feeling that there is a good reason David removes them. What
> would it break?
Nothing that I can think of easily. Perl has no problem with character
data, in general. It could be that, depending on where the CR characters
are, certain MIME decodings go wrong that otherwise wouldn't go wrong.
But that's rather unlikely.
--
#!perl -wpl # mmfppfmpmmpp mmpffm <[EMAIL PROTECTED]>
$p=3-2*/[^\W\dmpf_]/i;s.[a-z]{$p}.vec($f=join('',$p-1?chr(sub{$_[0]*9+$_[1]*3+
$_[2]}->(map{/p|f/i+/f/i}split//,$&)+97):qw(m p f)[map{((ord$&)%32-1)/$_%3}(9,
3,1)]),5,1)='`'lt$&;$f.eig;# Jan-Pieter Cornet
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Bare returns in message body
So, why does Mimedefang remove the return characters as it creates INPUTMSG? We'd really like to know where they are so we can track down bugs in our own users' mail software. We're interested in patching mimedefang.c to do this, but have this funny feeling that there is a good reason David removes them. What would it break? Joe Brennan Columbia University Information Technology ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
