Re: Test for empty EnvelopeFrom
On Thu, 24 Sep 2015, Reindl Harald wrote: Am 23.09.2015 um 19:24 schrieb Philip Prindeville: Stating facts here, not giving an opinion. Not sure what’s up for debate. if it is empty it's <> aka Null-Sender and you really don't block that because you violating RFC's, block sane autoreplies usng it to prevent mail-loops and the subject indiactes one thing; you donät really understand how email works Rejecting messages based on their content PERIOD is violating the RFC’s. What’s your point? do what you want - a empty envelope from is not a sign of spam An empty envelope from by itself is not a spam sign but when combined with other characteristics of a message can be a good spam sign. For example almost everthing coming from outlook.com is locally generated messages (either new user created content or NDRs). User generated content has headers added to indicate that, NDRs have have headers added to indicate that. So take Null-Sender && sourced from outlook.com && has client-headers && not-have NDR-headers is a very good spam indicator. -- Dave Funk University of Iowa College of Engineering 319/335-5751 FAX: 319/384-0549 1256 Seamans Center Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527 #include Better is not better, 'standard' is better. B{
Re: Test for empty EnvelopeFrom
>> I never said it was. >> >> What I said was that when it’s coming from a server that doesn’t >> except inbound messages (and hence can’t generate bounces) THEN it’s >> a sign of Spam. >Since when does a server handling outbound traffic have to accept >inbound mail? >Any setup with more than a dozen boxes usually separates in/outbound traffic I know of many of my customers I filter for don't send their outbound mail back through my filters (even though I tell them to) so they could generate bounce messages and definitely are firewalled off from the Internet on inbound TCP 25. I am pretty sure that the majority of sending mail servers on the Internet don't accept inbound TCP 25 connections.
Re: Test for empty EnvelopeFrom
On 09/24/2015 06:17 PM, Philip Prindeville wrote: On Sep 24, 2015, at 4:12 AM, Reindl Harald wrote: Am 23.09.2015 um 19:24 schrieb Philip Prindeville: Stating facts here, not giving an opinion. Not sure what’s up for debate. if it is empty it's <> aka Null-Sender and you really don't block that because you violating RFC's, block sane autoreplies usng it to prevent mail-loops and the subject indiactes one thing; you donät really understand how email works Rejecting messages based on their content PERIOD is violating the RFC’s. What’s your point? do what you want - a empty envelope from is not a sign of spam I never said it was. What I said was that when it’s coming from a server that doesn’t except inbound messages (and hence can’t generate bounces) THEN it’s a sign of Spam. Since when does a server handling outbound traffic have to accept inbound mail? Any setup with more than a dozen boxes usually separates in/outbound traffic
Re: Test for empty EnvelopeFrom
On Sep 24, 2015, at 4:12 AM, Reindl Harald wrote: > > > Am 23.09.2015 um 19:24 schrieb Philip Prindeville: >> Stating facts here, not giving an opinion. Not sure what’s up for debate. >>> >>> if it is empty it's <> aka Null-Sender and you really don't block that >>> because you violating RFC's, block sane autoreplies usng it to prevent >>> mail-loops and the subject indiactes one thing; you donät really understand >>> how email works >> >> Rejecting messages based on their content PERIOD is violating the RFC’s. >> What’s your point? > > do what you want - a empty envelope from is not a sign of spam > > I never said it was. What I said was that when it’s coming from a server that doesn’t except inbound messages (and hence can’t generate bounces) THEN it’s a sign of Spam.
Re: Test for empty EnvelopeFrom
On Thu, 24 Sep 2015 14:30:42 + David Jones wrote: > I agree with you and Reindl on this point too. I guess what I meant > to say is usually the hardest spam to block with a null sender is > backscatter from a normally trusted/good reputation mail server. Yes, that can be very annoying. Luckily, most of the big providers AFAIK use SPF and DKIM (and even DMARC) and can reject spoofed messages with a 5xx reply code instead of generating backscatter; since we've published SPF and DKIM records we've seen quite a drop in the backscatter coming our way. What backscatter we do get is usually from domains whose admins don't know or care about SPF, etc. > RBLs and SA with a well-trained Bayes DB do a very good job on new > emails with a null sender. Yes, they do. And Bayes is good even on some backscatter if it includes the original message. Regards, Dianne.
Re: Test for empty EnvelopeFrom
> >From: Dianne Skoll >Sent: Thursday, September 24, 2015 9:02 AM >To: users@spamassassin.apache.org >Subject: Re: Test for empty EnvelopeFrom >On Thu, 24 Sep 2015 12:21:33 + >David Jones wrote: >> I agree with Reindl. You can't block null senders or you break a lot >> of legit emails. >Well, if you run your own mail server, you can do whatever you like so >long as you accept the consequences. >I would say: A null sender is not necessarily the sign of spam, but it's >also not necessarily the sign of ham. We see a continuous background >chatter of spam messages that have a null envelope sender. And these >are new messages, not backscatter in response to anything. I agree with you and Reindl on this point too. I guess what I meant to say is usually the hardest spam to block with a null sender is backscatter from a normally trusted/good reputation mail server. RBLs and SA with a well-trained Bayes DB do a very good job on new emails with a null sender. >What *is* a very reliable spam indicator (and is a SpamAssassin rule >DSN_NO_MIMEVERSION) is mail from a null sender that lacks a >MIME-Version: header. Almost all auto-generated responses have that >header; a fair bit of null-sender spam does not. Good info. I checked my MailScanner logs and there are a lot of hits on this rule along with an invalid watermark so they seem to be closely related. I do see a number of Yahoo.com legit DSNs that seem to be hitting this rule (not surprised) but have a valid MS watermark. Dave >Regards, >Dianne.
Re: Test for empty EnvelopeFrom
On Thu, 24 Sep 2015 12:21:33 + David Jones wrote: > I agree with Reindl. You can't block null senders or you break a lot > of legit emails. Well, if you run your own mail server, you can do whatever you like so long as you accept the consequences. I would say: A null sender is not necessarily the sign of spam, but it's also not necessarily the sign of ham. We see a continuous background chatter of spam messages that have a null envelope sender. And these are new messages, not backscatter in response to anything. What *is* a very reliable spam indicator (and is a SpamAssassin rule DSN_NO_MIMEVERSION) is mail from a null sender that lacks a MIME-Version: header. Almost all auto-generated responses have that header; a fair bit of null-sender spam does not. Regards, Dianne.
Re: Test for empty EnvelopeFrom
On Thu, 24 Sep 2015 12:21:33 + David Jones wrote: > > >From: Reindl Harald > >do what you want - a empty envelope from is not a sign of spam > > I agree with Reindl. You can't block null senders or you break a lot > of legit emails. You're agreeing with his response to a conclusion he jumped to. The OP never said he wanted to block it, or even score it unconditionally. > There are tools out there to prevent backscatter so you can detect > fake messages with a null sender. It's not about backscatter, a lot of spam has no envelope sender address. The main problem with trying to score it is not avoiding FPs on legitimate DSNs and out-of-office replies, it's that a lot of legitimate bulk and autogenerated email has no envelope sender address.
Re: Test for empty EnvelopeFrom
>From: Reindl Harald >Sent: Thursday, September 24, 2015 5:12 AM >To: Philip Prindeville >Cc: users@spamassassin.apache.org >Subject: Re: Test for empty EnvelopeFrom >Am 23.09.2015 um 19:24 schrieb Philip Prindeville: >> Stating facts here, not giving an opinion. Not sure what’s up for debate. >>> >>> if it is empty it's <> aka Null-Sender and you really don't block that >>> because >>> you violating RFC's, block sane autoreplies usng it to prevent mail-loops >>> and >>> the subject indiactes one thing; you donät really understand how email works > >> Rejecting messages based on their content PERIOD is violating the RFC’s. >> What’s your point? >do what you want - a empty envelope from is not a sign of spam I agree with Reindl. You can't block null senders or you break a lot of legit emails. There are tools out there to prevent backscatter so you can detect fake messages with a null sender. MailScanner has this built in and works well if you send your outbound messages through it. It adds a special header that will be included in a legit bounce message but won't exist in a fake one so you can detect them. P.S. MailScanner can be a little tough to get going with the web interface. This is a VM with everything ready to roll with a number of excellent tweaks already done for you including the Watermarking feature to help block backscatter. http://efa-project.org/
Re: Test for empty EnvelopeFrom
Am 23.09.2015 um 19:24 schrieb Philip Prindeville: Stating facts here, not giving an opinion. Not sure what’s up for debate. if it is empty it's <> aka Null-Sender and you really don't block that because you violating RFC's, block sane autoreplies usng it to prevent mail-loops and the subject indiactes one thing; you donät really understand how email works Rejecting messages based on their content PERIOD is violating the RFC’s. What’s your point? do what you want - a empty envelope from is not a sign of spam signature.asc Description: OpenPGP digital signature
Re: Test for empty EnvelopeFrom
On Sep 23, 2015, at 6:35 AM, RW wrote: > On Tue, 22 Sep 2015 11:43:18 -0600 > Philip Prindeville wrote: > >> Hi. >> >> I?m using SA with MdF on Linux (Fedora 22). >> >> MdF generates the header ?Return-Path: ? for me, so that >> should be available to me in the rules. >> >> To test this, I wrote a couple of rules: >> >> header __L_EMPTY_SENDER EnvelopeFrom:addr !~ /./ >> header __L_MATCH_SENDER EnvelopeFrom:addr =~ /.*/ > > I think you're going to kick yourself. ".*" ,means zero or more > characters, so matches anything. > > It looks like the superfluous "*" is the only thing wrong here. No, I wanted to match the entirety of the Return-Path to see what it contained, and that SA was parsing it correctly… Using -D would allow me to see what the rule held as the match-string… except that wasn’t working due to bug 6360. Thanks Martin for fixing that so quickly! Just wanted to make sure that it wasn’t something silly like including a leading space, for example. > > >> What is a negative match, anyway? > > AFAIK it just means that the rule matched without matching any actual > text for the debug to display. > > >> Am I seeing https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360 >> in this case? > > This looks to be a bug where "negative match" is also displayed if the > matched text is "0", i.e. the number zero rather than a null string. Yeah, exactly. Or if you’re matching against the empty string… I.e.: EnvelopeFrom:addr =~ /^$/ -Philip
Re: Test for empty EnvelopeFrom
On Sep 22, 2015, at 12:58 PM, Reindl Harald wrote: > > > Am 22.09.2015 um 19:43 schrieb Philip Prindeville: >> I’m using SA with MdF on Linux (Fedora 22). >> >> MdF generates the header “Return-Path: ” for me, so that should >> be available to me in the rules. >> >> To test this, I wrote a couple of rules: >> >> header __L_EMPTY_SENDER EnvelopeFrom:addr !~ /./ >> header __L_MATCH_SENDER EnvelopeFrom:addr =~ /.*/ > > sorry, but you need to understand what the envelope-from is > > hint: it exists *always* because it's the "MAIL FROM" command long before > data and don't depend on headers, nice when your MTA adds a Envelope-Header, > normally it's he Return-Path and the sender *do not have* any business to > deal with that No one said the sender had anything to do with that. I was pointing out that SpamAssassin seems to extract the EnvelopeFrom either from the Return-Path: or a Envelope-Sender= or Envelope-From= line at the top of the message (in parse_received_line() in Mail::SpamAssassin:Message::Metadata::Received)… In this case, MdF is generating the first (if you don’t believe me, look in spam_assassin_mail() in /usr/bin/mimedefang.pl). Stating facts here, not giving an opinion. Not sure what’s up for debate. > > if it is empty it's <> aka Null-Sender and you really don't block that > because you violating RFC's, block sane autoreplies usng it to prevent > mail-loops and the subject indiactes one thing; you donät really understand > how email works Rejecting messages based on their content PERIOD is violating the RFC’s. What’s your point? If you think that no one is violating the RFC’s to spoof identities, obscure the origin of a message, etc. then you’re the one not understanding how email works in reality. > > https://en.wikipedia.org/wiki/Bounce_message > So what you’re saying is, since no one should block the null sender (i.e. the bounce message), it’s never going to be abused by anyone generating Spam… For instance, a host which never ACCEPTS incoming messages and therefore would never send a BOUNCE back in response to a message it received would never GENERATE a Spam with a null sender… Yet that’s exactly what’s happening to us. We’re seeing messages like: Return-Path: <> Received: from APC01-PU1-obe.outbound.protection.outlook.com (mail-pu1apc01hn0234.outbound.protection.outlook.com [104.47.126.234]) by mail.redfish-solutions.com (8.14.9/8.14.9) with ESMTP id t8HHb88M003095 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=OK) for ; Thu, 17 Sep 2015 11:37:14 -0600 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=<>; Received: from [100.66.17.62] (116.202.32.68) by SIXPR01MB0400.apcprd01.prod.exchangelabs.com (10.160.240.141) with Microsoft SMTP Server (TLS) id 15.1.274.11; Thu, 17 Sep 2015 17:36:52 + Content-Type: multipart/alternative; boundary="===0013032214==" MIME-Version: 1.0 Subject: Your trust … It’s notable that *.outbound.protection.outlook.come will NEVER generate a bounce, and hence never generate a null-sender, since those are handled by the inbound servers. But apparently I should accept these anyway, even though they represent an impossible scenario. Okay, thanks for the great advice. Next! -Philip
Re: Test for empty EnvelopeFrom
On Tue, 22 Sep 2015 11:43:18 -0600 Philip Prindeville wrote: > Hi. > > I?m using SA with MdF on Linux (Fedora 22). > > MdF generates the header ?Return-Path: ? for me, so that > should be available to me in the rules. > > To test this, I wrote a couple of rules: > > header __L_EMPTY_SENDER EnvelopeFrom:addr !~ /./ > header __L_MATCH_SENDER EnvelopeFrom:addr =~ /.*/ I think you're going to kick yourself. ".*" ,means zero or more characters, so matches anything. It looks like the superfluous "*" is the only thing wrong here. > What is a negative match, anyway? AFAIK it just means that the rule matched without matching any actual text for the debug to display. > Am I seeing https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360 > in this case? This looks to be a bug where "negative match" is also displayed if the matched text is "0", i.e. the number zero rather than a null string.
Re: Test for empty EnvelopeFrom
Am 22.09.2015 um 19:43 schrieb Philip Prindeville: I’m using SA with MdF on Linux (Fedora 22). MdF generates the header “Return-Path: ” for me, so that should be available to me in the rules. To test this, I wrote a couple of rules: header __L_EMPTY_SENDER EnvelopeFrom:addr !~ /./ header __L_MATCH_SENDER EnvelopeFrom:addr =~ /.*/ sorry, but you need to understand what the envelope-from is hint: it exists *always* because it's the "MAIL FROM" command long before data and don't depend on headers, nice when your MTA adds a Envelope-Header, normally it's he Return-Path and the sender *do not have* any business to deal with that if it is empty it's <> aka Null-Sender and you really don't block that because you violating RFC's, block sane autoreplies usng it to prevent mail-loops and the subject indiactes one thing; you donät really understand how email works https://en.wikipedia.org/wiki/Bounce_message signature.asc Description: OpenPGP digital signature
Test for empty EnvelopeFrom
Hi. I’m using SA with MdF on Linux (Fedora 22). MdF generates the header “Return-Path: ” for me, so that should be available to me in the rules. To test this, I wrote a couple of rules: header __L_EMPTY_SENDER EnvelopeFrom:addr !~ /./ header __L_MATCH_SENDER EnvelopeFrom:addr =~ /.*/ I’ve also tried: header __L_EMPTY_SENDER EnvelopeFrom:addr =~ /^$/ but in all cases, I get: Sep 22 11:07:46.237 [26384] dbg: rules: ran header rule __L_EMPTY_SENDER ==> got hit: "negative match" Sep 22 11:07:46.237 [26384] dbg: rules: ran header rule __L_MATCH_SENDER ==> got hit: "negative match” which I don’t get, because other similar rules end up showing what $& (or ${^MATCH}) would be (i.e. what matched). What is a negative match, anyway? Looking at the code, it seems that ::hit_rule_plugin_code() only gets called when we matched anyway, so saying it’s a negative match is counter-intuitive. Am I seeing https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6360 in this case? And comment 2 says: trunk (3.4.0): Bug 6360 : "negative match" on a "0" string - not fixed, appears cosmetic, just added a comment Sending lib/Mail/SpamAssassin/Plugin/Check.pm Committed revision 1338300. but the bug is marked “RESOLVED FIXED” so I’m confused. Should it be “WONTFIX” instead? Thanks, -Philip