Re: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[John Hardin]

On Thu, 17 Sep 2020, Kevin A. McGrail wrote:


sendgrid has seriously fallen from grace this year despite numerous
attempts to contact them and assist.

https://krebsonsecurity.com/2020/08/sendgrid-under-siege-from-hacked-accounts/
also sheds light on the issue too.


There's also a RBL for compromised sendgrid user IDs. See the thread 
starting at:


https://marc.info/?l=spamassassin-users&m=159803815425176&w=2


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.org pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  All I could think about was this bear is so close to me I can
  see its teeth. I could have kissed it. I wished I had a gun.
 -- Alyson Jones-Robinson
---
 Tomorrow: Talk Like a Pirate day

Re: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[Rob McEwen]

On 9/18/2020 6:38 AM, Loren Wilton wrote:
https://krebsonsecurity.com/2020/08/sendgrid-under-siege-from-hacked-accounts/ 


also sheds light on the issue too.


. SendGrid knows (or should konw) that it has compromised 
accounts. It could find out what some of them are for free by 
downloading Rob's list of 25 or so compromised accounts.



I strongly suspect that many of those accounts on our 2 Sendgrid lists 
are just plain 'ol spammers, NOT compromised. So some are compromised, 
some are spammers. And the list has grown to 594 SendGrid IDs 
(currently, as I type this) - much more than 25! Also, the list of 
domains found at the end of the SMTP-FROM that we're also deeming as 
spam or malicous has likewise grown to 87 domains


SEE:
https://www.invaluement.com/spdata/sendgrid-id-dnsbl.txt

AND:
https://www.invaluement.com/spdata/sendgrid-envelopefromdomain-dnsbl.txt
https://www.invaluement.com/spdata/sendgrid-envelopefromdomain-dnsbl-rbldnsd.txt
(2nd one formatted for rbldnsd)

I'm seeing evidence/reports that Sendgrid is likely using this data to 
greatly improve their system, and that this (maybe combined with their 
other efforts?) is finally starting to improve things? So that is good 
news. But I'm also shocked at how many hours go by where certain 
egregious accounts on our Sendgrid DNSBLs STILL stay in circulation 
while continuing to send spams, sometimes criminal phishing spams. But I 
also understand that they have to be careful about overly trusting 3rd 
party data, to ensure that they don't overreact to what might be an 
occasional false positive. It shouldn't be too long before they figure 
out that False Positives in those two Sendgrid lists are very very 
rare... practically non-existent. They probably should at least PAUSE 
campaigns pending further investigation. They should at least do that 
much, imo.


(They MIGHT also be suffering from the increasingly common and flawed 
view in the ESP industry - that not-illegal and CAN-SPAM-compliant mail 
is always legit and not spam - mistakenly not understanding that spam 
doesn't have to be illegal and malware, in order to be unsolicited and 
undesired by the recipient (aka "spam"). Maybe them seeing those types 
of accounts in our data is confusing them? I don't know - but much of 
the ESP industry is in great need of a "reset" - and this data is a good 
first step towards that!)


I was planning to spend much time this past week (1) adding this data to 
my own customer's direct query and rsync feeds, and (2) improving the 
instructions, including providing more specific instructions for adding 
this free version to various MTAs - but all that time got put into 
performance and effectiveness enhancements instead. Therefore, the data 
has greatly improved in just the past few days. New data sources were 
added into the mix - and many others of these spams these that were 
previously getting missed, are now getting caught - and the time from 
such a spam being first received - to that data getting into the list - 
has improved from about 1/2 a minute, to just a few seconds!


-- Rob McEwen invaluement.com

RE: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[Marc Roos]

 
But now it is Sendgrid tomorrow it is some other company, fact is were 
stuck with this trend of spammers outsourcing their spam trying to mix 
it with legitimate email. 

Legitimate clients are not aware of this and use these companies because 
of whatever ill advised reason. I am thinking about documenting this 
behaviour on 'my' hosting pages so people can read and be aware of this. 
I think if everyone does this, legitimate clients will stay away from 
these businesses. And if they stay away from these businesses, it is for 
'smaller' providers easier to manage (eg. blanket block the whole owned 
range)





-Original Message-
To: users@spamassassin.apache.org
Subject: Re: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

> https://krebsonsecurity.com/2020/08/sendgrid-under-siege-from-hacked-a
> ccounts/
> also sheds light on the issue too.

. SendGrid knows (or should konw) that it has compromised 
accounts. 
It could find out what some of them are for free by downloading Rob's 
list of 25 or so compromised accounts. It could find out what some of 
the other 400 are for $15 each, and could find out what some of the 
major offenders are for $400 each. Let's see, 400 compromised accounts 
times $400 is $16,000 dollars. SendGrid or Twillio can't afford a 
$16,000 cash outlay to find the account names of the major compromised 
accounts? Their head of security probably gets that much a month in 
salary and bonuses. It would be a trivial expense.

So what could they do once they knew which acocunts are compromised?
Are they helpless, and can only wring their hands and issue press 
releases saying They Have A Plan?

No. They can SHUT THE DAMN ACCOUNTS DOWN. Issue refunds to the owners if 
they feel generous. Tell the owners to open new accounts with 2FA.

But they won't do this, because they get their money from sending spam.

Loren

Re: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[Loren Wilton]

https://krebsonsecurity.com/2020/08/sendgrid-under-siege-from-hacked-accounts/
also sheds light on the issue too.


. SendGrid knows (or should konw) that it has compromised accounts. 
It could find out what some of them are for free by downloading Rob's list 
of 25 or so compromised accounts. It could find out what some of the other 
400 are for $15 each, and could find out what some of the major offenders 
are for $400 each. Let's see, 400 compromised accounts times $400 is $16,000 
dollars. SendGrid or Twillio can't afford a $16,000 cash outlay to find the 
account names of the major compromised accounts? Their head of security 
probably gets that much a month in salary and bonuses. It would be a trivial 
expense.


So what could they do once they knew which acocunts are compromised?
Are they helpless, and can only wring their hands and issue press releases 
saying They Have A Plan?


No. They can SHUT THE DAMN ACCOUNTS DOWN. Issue refunds to the owners if 
they feel generous. Tell the owners to open new accounts with 2FA.


But they won't do this, because they get their money from sending spam.

   Loren

Re: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[Kevin A. McGrail]

sendgrid has seriously fallen from grace this year despite numerous
attempts to contact them and assist.

https://krebsonsecurity.com/2020/08/sendgrid-under-siege-from-hacked-accounts/
also sheds light on the issue too.

Re: blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[Pedro David Marco]

 

   >On Thursday, September 17, 2020, 12:44:52 PM GMT+2, Marc Roos 
 wrote:  
 >For what it is worth. I was always under the impression that most of >hose 
 >companies that are using these networks known for 'harassing' 
>here just ignorant. I used to do business with the 'idiots' of 
>ucows/opensrs, trying to explain to them that it is not really wise to 
>end password reset emails via the same mail servers that their 'cheap 
>cients' are using for spamming. 

+1
We see quite oftnely companies sending valid invocies via free sendgrid 
accounts 

Pedreter  

blacklisting the likes of sendgrid, mailgun, mailchimp etc.

[Marc Roos]

 
For what it is worth. I was always under the impression that most of 
those companies that are using these networks known for 'harassing' 
where just ignorant. I used to do business with the 'idiots' of 
Tucows/opensrs, trying to explain to them that it is not really wise to 
send password reset emails via the same mail servers that their 'cheap 
clients' are using for spamming. 

However I just got email of a company medialab.co using this mailgun 
network. Turns out they had problems with getting blacklisted and that 
is why they moved there. So I tend to change my position, that it is 
quite legitimate to rate these networks as being bad by default. Maybe 
most clients just move there because they are sending shit. And now they 
can use the excuse that someone else caused the bad reputation.

Re: Blacklisting a stubborn sender

[Benny Pedersen]

Matus UHLAR - fantomas skrev den 2020-08-04 12:36:

Rupert Gallagher skrev den 2020-08-03 16:10:

The domains turn out to be already in the rfc-clueless.org database
since 2014.


On 04.08.20 02:32, Benny Pedersen wrote:

feel free to add it to spamassassin then

sadly 99% false possitives :/


do you mean, 99% of listings are incorrect?
Or just that 99% of hits are ham?


rfc clueless is imho unmaintained, it runs with stale data from 
rfc-ignorant, it needs love to be good again, and even if it was 
accurate 100% it will not help much on stoping spam

Re: Blacklisting a stubborn sender

[Matus UHLAR - fantomas]

Rupert Gallagher skrev den 2020-08-03 16:10:

The domains turn out to be already in the rfc-clueless.org database
since 2014.


On 04.08.20 02:32, Benny Pedersen wrote:

feel free to add it to spamassassin then

sadly 99% false possitives :/


do you mean, 99% of listings are incorrect?
Or just that 99% of hits are ham?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"They say when you play that M$ CD backward you can hear satanic messages."
"That's nothing. If you play it forward it will install Windows."

Re: Blacklisting a stubborn sender

[Benny Pedersen]

Rupert Gallagher skrev den 2020-08-03 16:10:

The domains turn out to be already in the rfc-clueless.org database
since 2014.


feel free to add it to spamassassin then

sadly 99% false possitives :/

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

The domains turn out to be already in the rfc-clueless.org database since 2014.

 Original Message 
On 1 Aug 2020, 14:58, Rupert Gallagher < r...@protonmail.com> wrote:
Two well known companies in my country persist in making the mistake of writing 
their mid with a non-public fqdn, violating the rfc. It has been so for the 
past three years, with me sending detailed, manually written error messages to 
their painstakingly collected admin addresses. Their answer is that everybody 
else accepts their invalid mid, and their servers are enterprise ibm / 
microsoft shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.

Is there an ombusdman that superintends the major blacklists and enforces rfc 
compliance through them?

Re: Blacklisting a stubborn sender

[@lbutlr]

On 02 Aug 2020, at 07:54, Kevin A. McGrail  wrote:
> If they aren't spending spam, why care about their MID or Helo format
> unless there is a delivery issue.

If they are sending mail with an invalid helo then it is perfectly valid to 
drop the connections. This may be a problem when you want to use this as an 
obvious spam fighting measure, but clueless gits misconfigure their mail 
servers and you then have to punch a hole through your perfectly reasonable 
anti-spam policy just to serve their cluelessness so that people get mail they 
want to get.

Every time you make your front-line defense weaker you increase the amount of 
spam you have to deal with to a great degree. Having to do this for someone is 
just incompetent is aggravating and expensive.

But, as always, you have to balance your aggravation against receiving the mail 
that the accounts on your server want to receive, and no one can measure that 
but the receiving mail server.

-- 
I gotta call my glitter guy

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

 Original Message 
On 2 Aug 2020, 17:02, Bill Cole < sausers-20150...@billmail.scconsult.com> 
wrote:

> if you want to authenticate email, ...

The helo is a necessary, but not sufficient criteria for authentication. I use 
them all, up to dane. However, they all fail with those two senders, because 
they do not implement them, and they send mail from an IP whose rDNS is that of 
the ISP. Authenticating the source in this case reduces to cheching the sending 
IP against those from known valid e-mails, as well all rfc compliance as 
evidence of some degree of home work from their side. I am talking about a bank 
and an accountancy firm, and I resent having to do this.

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

 Original Message 
On 2 Aug 2020, 17:02, Bill Cole < sausers-20150...@billmail.scconsult.com> 
wrote:

> smtpd_helo_restrictions

Good idea. Thank you.

Re: Blacklisting a stubborn sender

[Ralph Seichter]

* Bill Cole:

> Trusting the authenticity of email simply because it comes from a
> machine which uses a resolvable HELO in a particular domain is a naive
> approach unless you are *AT LEAST* using a DNS resolver that demands
> authenticated answers, i.e. requires DNSSEC [...]

Agreed, but I'd go one step further and call it dangerous instead of
just naive. Anything short of a verifiable, cryptographic signature
cannot be relied on when it comes to email authenticity. DNSSEC does not
provide protection against rogue email being sent from an organisation's
servers.

-Ralph

Re: Blacklisting a stubborn sender

[Ralph Seichter]

* Rupert Gallagher:

> They will procrastinate until the end of time unless we do something.

"We"? You are trying to drag others into your fight against
windmills. No thanks.

> I tried hard, but they are lazy/ignorant/careless. Blacklisting would
> trigger a problem with most of their customers, then they will try to
> de-list at first, then they will comply when de-listing is rejected.

Who died and made you the enforcer of email purity? Block locally if you
must, silly as it is in the case of a bank, but don't presume to be the
deciding factor on what others accept.

-Ralph

Re: Blacklisting a stubborn sender

[Ralph Seichter]

* Rupert Gallagher:

> Correction: it is not the mid, it is the helo.

/me snorts

But of course it is. :-D After I have just read your ramblings in a
2017 mailing list thread [1], in which you made the same nonsense
remarks about message IDs, why would I doubt you?

[1] 
https://readlist.com/lists/incubator.apache.org/spamassassin-users/20/101931.html

-Ralph

Re: Blacklisting a stubborn sender

[Ralph Seichter]

* RW:

> https://readlist.com/lists/incubator.apache.org/spamassassin-users/20/101951.html

Oh my, I was not aware of that. Looks like Rupert has nurtured his pet
peeve for at least three years. Thats a long time of being stubbornly
wrong.

-Ralph

Re: Blacklisting a stubborn sender

[Bill Cole]

On 2 Aug 2020, at 10:07, Rupert Gallagher wrote:

To ignore it, as you say, I would have to remove the postfix check, 
write rules to implement a non-blocking check, then write rules to 
implement the rejection except for whitelisted domains.


OR, in the language of Postfix configuration:

   smtpd_helo_required = yes
   smtpd_helo_restrictions = check_helo_access pcre:badheloallowed, 
reject_unknown_helo_hostname, reject_non_fqdn_helo_hostname


And put entries into $config_directory/badheloallowed like this:

   /localhost/  PERMIT
   /invalid_hostname/ PERMIT
   /unresolvable.rbs.co.uk/ PERMIT
   /mailhost.sc.com/ PERMIT


It is a lot of work,


I just did it for you, for free. The hardest "work" was looking up a 
couple of bank domains for examples.


to allow a bank and an accounting firm to violate an industry 
standard, and still have the doubt on the authenticity of their 
e-mails. No thank you.


If you want to authenticate email, it needs to use some form of internal 
authentication such as DKIM, S/MIME or OpenPGP. Trusting the 
authenticity of email simply because it comes from a machine which uses 
a resolvable HELO in a particular domain is a naive approach unless you 
are *AT LEAST* using a DNS resolver that demands authenticated answers, 
i.e. requires DNSSEC, treating non-DNSSEC replies as meaningless.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not For Hire (currently)

Re: Blacklisting a stubborn sender

[Bill Cole]

On 2 Aug 2020, at 9:18, Rupert Gallagher wrote:

They will procrastinate until the end of time unless we do something. 
I tried hard, but they are lazy/ignorant/careless. Blacklisting would 
trigger a problem with most of their customers, then they will try to 
de-list at first, then they will comply when de-listing is rejected.


In what way does this relate to SpamAssassin?

You are free to block based on a non-resolving HELO domain or even a 
non-resolving M-ID domain, and you can use SA to do so. The mechanisms 
are documented. I'm sure you can get help here if the documentation 
isn't clear to you. For the HELO domain that is most efficiently done in 
your MTA and as Matus says: if you're using Postfix you can do that with 
the various reject_*_helo_hostname restrictions, depending on what sort 
of strictures you want to use. You can even lie to yourself about that 
being an "enforcement" of some RFC, even though no RFC blesses the 
rejection of mail based on any characteristic of the HELO domain or says 
that it MUST resolve. A non-resolving HELO name may or may not correlate 
to mail being spam, and if you can establish that correlation and define 
it in SA rules I'm sure we would all like to know that and I would be 
happy to test such rules in the project's RuleQA system.


If you want to recruit DNSBL operators to your point of view, this is 
not a reasonable forum for that campaign. They are mostly not here. If 
you want to recruit mail admins to your point of view, this is not a 
reasonable forum for that campaign. They are also mostly not here, even 
the ones who are consciously using SpamAssassin.


If there's no correlation of your pet HELO domain rules to whether or 
not email is spam then you will have a very hard time recruiting anyone 
to join your campaign by blocking mail, no matter where you find a 
suitable audience for that evangelism.




 Original Message 
On 2 Aug 2020, 12:30, Matus UHLAR - fantomas < uh...@fantomas.sk> 
wrote:

On 02.08.20 05:11, Rupert Gallagher wrote:

Correction: it is not the mid, it is the helo.

oh... this is something quite different.
But unless multiple servers start implementing 
reject_unknown_helo_hostname,

such companies ignore to change that...
... apparently with possibly reject_non_fqdn_elo_hostname and
reject_invalid_helo_hostname. and smtpd_helo_required=yes of course

 Original Message 
On 1 Aug 2020, 14:58, Rupert Gallagher < r...@protonmail.com> wrote:
Two well known companies in my country persist in making the mistake 
of writing their mid with a non-public fqdn, violating the rfc. It 
has been so for the past three years, with me sending detailed, 
manually written error messages to their painstakingly collected 
admin addresses. Their answer is that everybody else accepts their 
invalid mid, and their servers are enterprise ibm / microsoft 
shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many 
blacklists, and I have no intention to go through each idiosyncratic 
procedure.


Is there an ombusdman that superintends the major blacklists and 
enforces rfc compliance through them?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not For Hire (currently)

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

To ignore it, as you say, I would have to remove the postfix check, write rules 
to implement a non-blocking check, then write rules to implement the rejection 
except for whitelisted domains. It is a lot of work, to allow a bank and an 
accounting firm to violate an industry standard, and still have the doubt on 
the authenticity of their e-mails. No thank you.

 Original Message 
On 2 Aug 2020, 15:54, Kevin A. McGrail < kmcgr...@apache.org> wrote:
On 8/2/2020 9:18 AM, Rupert Gallagher wrote:
> They will procrastinate until the end of time unless we do something.
> I tried hard, but they are lazy/ignorant/careless. Blacklisting would
> trigger a problem with most of their customers, then they will try to
> de-list at first, then they will comply when de-listing is rejected.
If they aren't spending spam, why care about their MID or Helo format
unless there is a delivery issue.
This project exists to stop spam not to weaponize our capabilities to
become the RFC-police.
If the users have consent to receive the mail and the mail is getting to
the right person, I'd recommend you ignore it.
Regards,
KAM
--
Kevin A. McGrail
kmcgr...@apache.org
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Blacklisting a stubborn sender

[Kevin A. McGrail]

On 8/2/2020 9:18 AM, Rupert Gallagher wrote:
> They will procrastinate until the end of time unless we do something.
> I tried hard, but they are lazy/ignorant/careless. Blacklisting would
> trigger a problem with most of their customers, then they will try to
> de-list at first, then they will comply when de-listing is rejected. 

If they aren't spending spam, why care about their MID or Helo format
unless there is a delivery issue.

This project exists to stop spam not to weaponize our capabilities to
become the RFC-police.

If the users have consent to receive the mail and the mail is getting to
the right person, I'd recommend you ignore it.

Regards,

KAM

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

Re: Blacklisting a stubborn sender

[Matus UHLAR - fantomas]

On 02.08.20 13:18, Rupert Gallagher wrote:

They will procrastinate until the end of time unless we do something.  I
tried hard, but they are lazy/ignorant/careless.  Blacklisting would
trigger a problem with most of their customers, then they will try to
de-list at first, then they will comply when de-listing is rejected.


I don't think there's point in blacklisting hoat that sends fake helo, when
you can block the helo itself.

yes, I think that such helo should be blocked and I block it wherever I can.


 Original Message 
On 2 Aug 2020, 12:30, Matus UHLAR - fantomas < uh...@fantomas.sk> wrote:
On 02.08.20 05:11, Rupert Gallagher wrote:

Correction: it is not the mid, it is the helo.

oh... this is something quite different.
But unless multiple servers start implementing reject_unknown_helo_hostname,
such companies ignore to change that...
... apparently with possibly reject_non_fqdn_elo_hostname and
reject_invalid_helo_hostname. and smtpd_helo_required=yes of course

 Original Message 
On 1 Aug 2020, 14:58, Rupert Gallagher < r...@protonmail.com> wrote:
Two well known companies in my country persist in making the mistake of writing 
their mid with a non-public fqdn, violating the rfc. It has been so for the 
past three years, with me sending detailed, manually written error messages to 
their painstakingly collected admin addresses. Their answer is that everybody 
else accepts their invalid mid, and their servers are enterprise ibm / 
microsoft shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.

Is there an ombusdman that superintends the major blacklists and enforces rfc 
compliance through them?


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Enter any 12-digit prime number to continue.

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

They will procrastinate until the end of time unless we do something. I tried 
hard, but they are lazy/ignorant/careless. Blacklisting would trigger a problem 
with most of their customers, then they will try to de-list at first, then they 
will comply when de-listing is rejected.

 Original Message 
On 2 Aug 2020, 12:30, Matus UHLAR - fantomas < uh...@fantomas.sk> wrote:
On 02.08.20 05:11, Rupert Gallagher wrote:
>Correction: it is not the mid, it is the helo.
oh... this is something quite different.
But unless multiple servers start implementing reject_unknown_helo_hostname,
such companies ignore to change that...
... apparently with possibly reject_non_fqdn_elo_hostname and
reject_invalid_helo_hostname. and smtpd_helo_required=yes of course
> Original Message 
>On 1 Aug 2020, 14:58, Rupert Gallagher < r...@protonmail.com> wrote:
>Two well known companies in my country persist in making the mistake of 
>writing their mid with a non-public fqdn, violating the rfc. It has been so 
>for the past three years, with me sending detailed, manually written error 
>messages to their painstakingly collected admin addresses. Their answer is 
>that everybody else accepts their invalid mid, and their servers are 
>enterprise ibm / microsoft shitware that they are unwilling to fix. Since we 
>get a lot of their emails, I decided to scale up their problem. There are many 
>blacklists, and I have no intention to go through each idiosyncratic procedure.
>
>Is there an ombusdman that superintends the major blacklists and enforces rfc 
>compliance through them?
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod

Re: Blacklisting a stubborn sender

[Matus UHLAR - fantomas]

On 02.08.20 05:11, Rupert Gallagher wrote:

Correction: it is not the mid, it is the helo.


oh... this is something quite different.
But unless multiple servers start implementing reject_unknown_helo_hostname,
such companies ignore to change that...

... apparently with possibly reject_non_fqdn_elo_hostname and
reject_invalid_helo_hostname. and smtpd_helo_required=yes of course



 Original Message 
On 1 Aug 2020, 14:58, Rupert Gallagher < r...@protonmail.com> wrote:
Two well known companies in my country persist in making the mistake of writing 
their mid with a non-public fqdn, violating the rfc. It has been so for the 
past three years, with me sending detailed, manually written error messages to 
their painstakingly collected admin addresses. Their answer is that everybody 
else accepts their invalid mid, and their servers are enterprise ibm / 
microsoft shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.

Is there an ombusdman that superintends the major blacklists and enforces rfc 
compliance through them?


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

Correction: it is not the mid, it is the helo.

 Original Message 
On 1 Aug 2020, 14:58, Rupert Gallagher < r...@protonmail.com> wrote:
Two well known companies in my country persist in making the mistake of writing 
their mid with a non-public fqdn, violating the rfc. It has been so for the 
past three years, with me sending detailed, manually written error messages to 
their painstakingly collected admin addresses. Their answer is that everybody 
else accepts their invalid mid, and their servers are enterprise ibm / 
microsoft shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.

Is there an ombusdman that superintends the major blacklists and enforces rfc 
compliance through them?

Re: Blacklisting a stubborn sender

[RW]

On Sat, 01 Aug 2020 18:48:04 +0200
Ralph Seichter wrote:

> * Rupert Gallagher:
> 
> > They have explicit consent to send rfc compliant e-mail.  
> 
> "They" are not violating RFC 2822 with their message IDs, as I already
> explained in message <87eeoqfpel@wedjat.horus-it.com>.
> 
> > Rfc-clueless.org seems.a good starting point.  
> 
> You are the one who misunderstands the relevant section of RFC 2822.
> Why would you want to report that to an even wider audience?

Not for the first time either:

https://readlist.com/lists/incubator.apache.org/spamassassin-users/20/101951.html

Re: Blacklisting a stubborn sender

[Bill Cole]

On 1 Aug 2020, at 8:58, Rupert Gallagher wrote:

Two well known companies in my country persist in making the mistake 
of writing their mid with a non-public fqdn, violating the rfc.


As Ralph says, that is not a violation of any RFC. There is no "MUST" 
condition in 5322 or its 2 most recent predecessors referencing the use 
of or resolvability of a domain in the RHS of a Message-ID. A domain is 
RECOMMENDED, but there's no mention of its public resolvability


It has been so for the past three years, with me sending detailed, 
manually written error messages to their painstakingly collected admin 
addresses. Their answer is that everybody else accepts their invalid 
mid, and their servers are enterprise ibm / microsoft shitware that 
they are unwilling to fix. Since we get a lot of their emails, I 
decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.


Is there an ombusdman that superintends the major blacklists and 
enforces rfc compliance through them?


Of course there isn't. If there was someone who could tell DNSBL 
operators what addresses to list or not list, there wouldn't be hundreds 
of independent DNSBLs.


Also, the idea that RFC compliance is something that can or should be 
"enforced" is entirely inconsistent with the basic principles of what 
RFCs are. They are documentation, not law.



--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not For Hire (currently)

Re: Blacklisting a stubborn sender

[Ralph Seichter]

* Rupert Gallagher:

> They have explicit consent to send rfc compliant e-mail.

"They" are not violating RFC 2822 with their message IDs, as I already
explained in message <87eeoqfpel@wedjat.horus-it.com>.

> Rfc-clueless.org seems.a good starting point.

You are the one who misunderstands the relevant section of RFC 2822.
Why would you want to report that to an even wider audience?

-Ralph

Re: Blacklisting a stubborn sender

[Rupert Gallagher]

They have explicit consent to send rfc compliant e-mail. Rfc-clueless.org 
seems.a good starting point.

Thank you

 Original Message 
On 1 Aug 2020, 15:53, Kevin A. McGrail < kmcgr...@apache.org> wrote:
On Sat, Aug 1, 2020 at 8:59 AM Rupert Gallagher  wrote:
Two well known companies in my country persist in making the mistake of writing 
their mid with a non-public fqdn, violating the rfc. It has been so for the 
past three years, with me sending detailed, manually written error messages to 
their painstakingly collected admin addresses. Their answer is that everybody 
else accepts their invalid mid, and their servers are enterprise ibm / 
microsoft shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.

Is there an ombusdman that superintends the major blacklists and enforces rfc 
compliance through them?

First, I use Chris Santerre's definition of Spam that spam is about not having 
consent rather than the content. If someone sends something with RFC issues to 
evade spam detection, that demonstrates a lack of consent and clear intent. But 
if it's something with consent, as long as the email can get from point A to 
point B, bending the RFCs is fine. I'd equate it to a kid addressing an 
envelope to his Granny in crayon, forgetting the zip code and put the stamp on 
the wrong corner. The post office can figure it out automatically and it gets 
where it's meant to go. By comparison, the big bad Wolf trying to contact 
Granny from his jail cell would be spam. The RFC compliance might be an 
indicator for me but not a reason to block but that's my take. If you want to 
share a sample on pastebin, I can give you my take on it using the content vs 
consent litmus test for rules/blocking.

Second, I'm not aware of any ombudsman for RBLs. Many RBLs are run by distinct 
organizations and many have unique listing/delisting criteria. An ombudsman 
would basically be a consultant spending a lot of time on the issue so you 
might just go that route and hire someone.

Finally, there are RBLs you might want to use. There used to be RFC Ignorant 
but it appears to be shuttered. Take a look at http://rfc-clueless.org/

Also if they use the same domain, just locally block it.

Regards,
KAM

Re: Blacklisting a stubborn sender

[Kevin A. McGrail]

On Sat, Aug 1, 2020 at 8:59 AM Rupert Gallagher  wrote:

> Two well known companies in my country persist in making the mistake of
> writing their mid with a non-public fqdn, violating the rfc. It has been so
> for the past three years, with me sending detailed, manually written error
> messages to their painstakingly collected admin addresses. Their answer is
> that everybody else accepts their invalid mid, and their servers are
> enterprise ibm / microsoft shitware that they are unwilling to fix. Since
> we get a lot of their emails, I decided to scale up their problem. There
> are many blacklists, and I have no intention to go through each
> idiosyncratic procedure.
>
> Is there an ombusdman that superintends the major blacklists and enforces
> rfc compliance through them?
>

First, I use Chris Santerre's definition of Spam that spam is about not
having consent rather than the content.  If someone sends something with
RFC issues to evade spam detection, that demonstrates a lack of consent and
clear intent.  But if it's something with consent, as long as the email can
get from point A to point B, bending the RFCs is fine.  I'd equate it to a
kid addressing an envelope to his Granny in crayon, forgetting the zip code
and put the stamp on the wrong corner.  The post office can figure it out
automatically and it gets where it's meant to go.  By comparison, the big
bad Wolf trying to contact Granny from his jail cell would be spam.  The
RFC compliance might be an indicator for me but not a reason to block but
that's my take.  If you want to share a sample on pastebin, I can give you
my take on it using the content vs consent litmus test for rules/blocking.

Second, I'm not aware of any ombudsman for RBLs.  Many RBLs are run by
distinct organizations and many have unique listing/delisting criteria.  An
ombudsman would basically be a consultant spending a lot of time on the
issue so you might just go that route and hire someone.

Finally, there are RBLs you might want to use.  There used to be RFC
Ignorant but it appears to be shuttered.  Take a look at
http://rfc-clueless.org/

Also if they use the same domain, just locally block it.

Regards,
KAM

Re: Blacklisting a stubborn sender

[Ralph Seichter]

* Rupert Gallagher:

> Two well known companies in my country persist in making the mistake
> of writing their mid with a non-public fqdn, violating the rfc. [...]
> been so for the past three years, with me sending detailed, manually
> Their answer is that everybody else accepts their invalid mid, and
> their servers are enterprise ibm / microsoft shitware that they are
> unwilling to fix.

RFC 2822 (https://tools.ietf.org/html/rfc2822#section-3.6.4) does not
state that a "public fqdn" (or any FQDN) is required in the message ID.
Formally, the id-right portion can be any dot-atom-text. Also, the
Message-ID header itself is optional, although recommended.

In other words, you are the stubborn one, and wrong to boot. ;-)

-Ralph

Blacklisting a stubborn sender

[Rupert Gallagher]

Two well known companies in my country persist in making the mistake of writing 
their mid with a non-public fqdn, violating the rfc. It has been so for the 
past three years, with me sending detailed, manually written error messages to 
their painstakingly collected admin addresses. Their answer is that everybody 
else accepts their invalid mid, and their servers are enterprise ibm / 
microsoft shitware that they are unwilling to fix. Since we get a lot of their 
emails, I decided to scale up their problem. There are many blacklists, and I 
have no intention to go through each idiosyncratic procedure.

Is there an ombusdman that superintends the major blacklists and enforces rfc 
compliance through them?

Re: IP Blacklisting

[Benny Pedersen]

Axb skrev den 2013-07-12 13:48:


Google for rbldnsd - this is outside of SA's scope.


if users begin googleing maybe some finds this one:

http://mail-archives.apache.org/mod_mbox/spamassassin-users/201103.mbox/%3calpine.deb.2.00.1103141313230.2...@pyxis.theca-tabellaria.de%3E

Re: IP Blacklisting

[Benny Pedersen]

Moein Sarvi skrev den 2013-07-12 13:43:

I want to use a mechanism that can be done by shell programming to
add remove daily IP address automatically
 my goal is  reject some IP addresses and rise up score of some other
IP sometimes as well.


make shell scripts that maintain sql ip blacklists, then use that sql 
table with regulary sql maps in postfix, saves rebuild reload maps when 
its setup

Re: IP Blacklisting

[Benny Pedersen]

Simon Loewenthal skrev den 2013-07-12 12:11:

If you use Postfix for your MTA, then drop into your_ header_checks_ 
file


or better make a cidr map file:

# cat cidr.map
192.168.1.0/24 REJECT
127.0.0.0/8 DUNNO

# in main.cf
smtpd_client_restrictions=
 ...
 check_client_access cidr:/path/to/cidr.map
 ...

note spaces in main.cf

depending on header_checks is next best option

Re: IP Blacklisting

[Benny Pedersen]

Moein Sarvi skrev den 2013-07-12 02:44:


is there anyway to blacklist an IP address?


nope, spamassassin does not block, if you want ip blocked do in mta 
stage


all spamassassin can do is to score and add headers

Re: IP Blacklisting

[Karsten Bräckelmann]

On Fri, 2013-07-12 at 13:22 +0430, Moein Sarvi wrote:
> First of all thanks for your great answer,

Please DO KEEP the thread on-list, and ONLY follow-up privately if you
really mean to. I am not the only one who can answer your questions.

> I wanna know both situation, I mean rejecting an IP address in the
> recieve step and sometimes punish some IP address and rise it's score,

I suggest to carefully re-read my post.

As I clearly mentioned, rejecting at the SMTP stage is outside the scope
of SA, and actually is a layer earlier. Generally it seems most of your
questions could already be answered by googling and reading some
documentation.

As for the SA part and "punishing" (aka scoring) based on sender IP
address, I mentioned some likely candidates for rule writing. However,
the best solution depends -- which brings me to the next part I can only
reiterate from my previous post:

What exactly is the problem you're facing and the result you want to
achieve?


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: IP Blacklisting

[Axb]

On 07/12/2013 01:43 PM, Moein Sarvi wrote:

I want to use a mechanism that can be done by shell programming to add
remove daily IP address automatically
my goal is  reject some IP addresses and rise up score of some other IP
sometimes as well.



Google for rbldnsd - this is outside of SA's scope.

Re: IP Blacklisting

[Moein Sarvi]

I want to use a mechanism that can be done by shell programming to add
remove daily IP address automatically
my goal is  reject some IP addresses and rise up score of some other IP
sometimes as well.

Re: IP Blacklisting

[Simon Loewenthal]

 

On 2013-07-12 9:02, Karsten Bräckelmann wrote: 

> On Fri, 2013-07-12 at 05:14 +0430, Moein Sarvi wrote:
> 
>> Hello is there anyway to blacklist an IP address?
> 
> Yes. Step 1: Create your own blacklist. Step 2: Report the IP. Optional
> step 3: Create rules in SA to query your blacklist created in step 1. Of
> course, I am merely assuming here you are actually asking something
> relevant to SA...
> 
> Joking apart, your question is *really* vague. In cases like this, it is
> a lot better to describe your actual problem, rather than asking
> something this broad. You still can add the missing info, and tell us
> about your problem.
> 
> Bunch-o-pointers regarding "blacklisting" an IP address:
> 
> SA does not reject, quarantine, drop or deliver mail. All it does is
> scoring. Thus, in case your "blacklisting" query involves these, you'd
> better check back with your SA calling layer.
> 
> If you definitely are about rejecting mail from a given IP, you'd want
> to look at your MX STMP configuration.
> 
> If you are happy to "severely punish" mail sent from a given IP, without
> a need to reject the mail, SA can do what you want. Punishment ranges
> from scoring, classifying as spam, all the way up to quarantining and
> simply dropping down the bin bucket -- the latter two depending on the
> following tools in your mail-processing chain.
> 
> Flooring mail in SA sent via a given IP (aka blacklisting) is possible
> in various ways, depending on your needs, configuration, accuracy of
> your configuration (like receiving mail via mailing lists) -- and of
> course your knowledge of mail headers, SA rules, SA pseudo headers, and
> RE in general. But I digress...
> 
> Likely candidates are the X-Spam-Relays-* Untrusted and External pseudo
> headers. But that could be done more efficiently in your SMTP, if you
> mean *black* as a pseudonym of *block*.
> 
> And if you really dislike the IP, you could als craft some simple
> Received header rules in SA. Though at this point, we're crossing the
> line between blacklist and blacklist. And deep header parsing.
> 
> Where did I start off again? Oh, right -- what exactly is the problem
> you're facing and the result you want to achieve?

Hi, 

Perhaps: 

header BLACKLIST_IP Received=~ /[IPaddress]/
 score BLACKLIST_IP 100
 describe BLACKLIST_IP Disallow from IP address 

If you use Postfix for your MTA, then drop into your_ header_checks_
file 

/^Received: from IPaddress/ REJECT Bye bye to your IP address

and then and add into the_ main.cf_ 

header_checks = pcre:/etc/postfix/header_checks 

Completely untested and not really thought about, of course. I suspect
my regexes are broken, but this gives you an idea. 

-- 
"I decided that I was a lemon for a couple of weeks. I kept myself
amused all that time jumping in and out of a gin and tonic."
simon@klunky .co.uk / .org
 

Re: IP Blacklisting

[Karsten Bräckelmann]

On Fri, 2013-07-12 at 05:14 +0430, Moein Sarvi wrote:
> Hello
> is there anyway to blacklist an IP address?

Yes. Step 1: Create your own blacklist. Step 2: Report the IP. Optional
step 3: Create rules in SA to query your blacklist created in step 1. Of
course, I am merely assuming here you are actually asking something
relevant to SA...

Joking apart, your question is *really* vague. In cases like this, it is
a lot better to describe your actual problem, rather than asking
something this broad. You still can add the missing info, and tell us
about your problem.


Bunch-o-pointers regarding "blacklisting" an IP address:

SA does not reject, quarantine, drop or deliver mail. All it does is
scoring. Thus, in case your "blacklisting" query involves these, you'd
better check back with your SA calling layer.

If you definitely are about rejecting mail from a given IP, you'd want
to look at your MX STMP configuration.

If you are happy to "severely punish" mail sent from a given IP, without
a need to reject the mail, SA can do what you want. Punishment ranges
from scoring, classifying as spam, all the way up to quarantining and
simply dropping down the bin bucket -- the latter two depending on the
following tools in your mail-processing chain.

Flooring mail in SA sent via a given IP (aka blacklisting) is possible
in various ways, depending on your needs, configuration, accuracy of
your configuration (like receiving mail via mailing lists) -- and of
course your knowledge of mail headers, SA rules, SA pseudo headers, and
RE in general. But I digress...

Likely candidates are the X-Spam-Relays-* Untrusted and External pseudo
headers. But that could be done more efficiently in your SMTP, if you
mean *black* as a pseudonym of *block*.

And if you really dislike the IP, you could als craft some simple
Received header rules in SA. Though at this point, we're crossing the
line between blacklist and blacklist. And deep header parsing.


Where did I start off again? Oh, right -- what exactly is the problem
you're facing and the result you want to achieve?


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

IP Blacklisting

[Moein Sarvi]

Hello
is there anyway to blacklist an IP address?

Re: Blacklisting based on SPF

[Marc Perkel]

On 10/11/2011 6:49 AM, Matus UHLAR - fantomas wrote:

On 7 Oct 2011 00:28:49 -, John Levine wrote:

Nobody with any interest in delivering the mail that their users want.
The error rate is much, much too high.



On 10/7/2011 12:50 AM, Benny Pedersen wrote:

how ?


On 10.10.11 07:00, Marc Perkel wrote:
All forwarded email would fail SPF testing.  You would be blocking 
all hosted spam filtering services for example.


FUD and bullshit.

such forwarding will break SPF iff the forwarder does not change the 
mail from: address, and in such case it FAKES the return path, since 
it's not the original sender who sent the mail, it's the recipient.
Whoever wishes to get mail forwarded through mailbox that does not 
this kind of rewriting, should configure the forwarder as 
trusted/internal for this case.




http://www.openspf.org/FAQ/Forwarding



--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: Blacklisting based on SPF

[Matus UHLAR - fantomas]

On Wed, 12 Oct 2011 16:08:12 +0200, Matus UHLAR - fantomas wrote:

was this changed or you just continue FUDding?


On 12.10.11 16:18, Benny Pedersen wrote:

From: header is NOT envelope-from header, stop fuding self


From: is _NOT_ "mail from:" and since DKIM has nothing with mail from:, 
I don't see how could forwarding break DKIM, unless modifying message 
content (From: header) which I was not talking about.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
WinError #9: Out of error messages.

Re: Blacklisting based on SPF

[Benny Pedersen]

On Wed, 12 Oct 2011 16:08:12 +0200, Matus UHLAR - fantomas wrote:


was this changed or you just continue FUDding?


From: header is NOT envelope-from header, stop fuding self

Re: Blacklisting based on SPF

[Matus UHLAR - fantomas]

On Tue, 11 Oct 2011 17:14:06 +0200, Matus UHLAR - fantomas wrote:


(and possibly list of forwarders who do not rewrite mail from)


On 11.10.11 21:03, Benny Pedersen wrote:
breaks dkim, and instalations that use from: as envelope sender 
header ask for troubles


cite from rfc4686:

DKIM operates entirely on the content (body and selected header
fields) of the message, as defined in RFC 2822 [RFC2822].  The
transmission of messages via SMTP, defined in RFC 2821 [RFC2821], and
such elements as the envelope-from and envelope-to addresses and the
HELO domain are not relevant to DKIM verification.

was this changed or you just continue FUDding?

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Christian Science Programming: "Let God Debug It!".

Re: Blacklisting based on SPF

[Matus UHLAR - fantomas]

On Tue, 11 Oct 2011 15:49:36 +0200, Matus UHLAR - fantomas wrote:

such forwarding will break SPF iff the forwarder does not change the
mail from: address, and in such case it FAKES the return path, since
it's not the original sender who sent the mail, it's the recipient.


On 11.10.11 20:55, Benny Pedersen wrote:

it breaks dkim if anything is changed, this is not fud


Well,
- SPF is not DKIM
- DKIM is broken if someone changes the mail content, not the envelope 
  address.


according to some discussions the DKIM seems to have problems with mail 
reformatting by courier MTA. Maybe the specification could be relaxed 
to case insensitive checking of headers...



Whoever wishes to get mail forwarded through mailbox that does not
this kind of rewriting, should configure the forwarder as
trusted/internal for this case.


only trusted_network for the forwarding mta is needed to make spf work


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Silvester Stallone: Father of the RISC concept.

Re: Blacklisting based on SPF

[Benny Pedersen]

On Tue, 11 Oct 2011 17:14:06 +0200, Matus UHLAR - fantomas wrote:


(and possibly list of forwarders who do not rewrite mail from)


breaks dkim, and instalations that use from: as envelope sender header 
ask for troubles

Re: Blacklisting based on SPF

[Benny Pedersen]

On Tue, 11 Oct 2011 15:49:36 +0200, Matus UHLAR - fantomas wrote:

such forwarding will break SPF iff the forwarder does not change the
mail from: address, and in such case it FAKES the return path, since
it's not the original sender who sent the mail, it's the recipient.


it breaks dkim if anything is changed, this is not fud


Whoever wishes to get mail forwarded through mailbox that does not
this kind of rewriting, should configure the forwarder as
trusted/internal for this case.


only trusted_network for the forwarding mta is needed to make spf work

Re: Blacklisting based on SPF

[Matus UHLAR - fantomas]

On 05.10.11 11:01, Julian Yap wrote:

I've noticed some trojans with addresses from usps.com slip through.

Does anyone blacklist based on SPF?


According to SPF definition, all mail that fails SPF check, is forged 
and therefore it should be rejected (in case of FAIL result), or very 
carefully cheked.


In reality, there are problems related to
- mail forwarders who can't tag the mail as forwarded (and thus, they 
  in fact fake the envelope sender)
- misconfigured SPF and misconfigured mailers of companies who do 
  not understand the SPF principle and outsource the mailers outside


usually, people do what you want either by defining their own rule, 
but as it turns out, having something like SPF blacklist would be a 
good idea.


(and possibly list of forwarders who do not rewrite mail from)
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
LSD will make your ECS screen display 16.7 million colors

Re: Blacklisting based on SPF

[Matus UHLAR - fantomas]

On 7 Oct 2011 00:28:49 -, John Levine wrote:

Nobody with any interest in delivering the mail that their users want.
The error rate is much, much too high.



On 10/7/2011 12:50 AM, Benny Pedersen wrote:

how ?


On 10.10.11 07:00, Marc Perkel wrote:
All forwarded email would fail SPF testing.  You would be blocking 
all hosted spam filtering services for example.


FUD and bullshit.

such forwarding will break SPF iff the forwarder does not change the 
mail from: address, and in such case it FAKES the return path, since 
it's not the original sender who sent the mail, it's the recipient. 

Whoever wishes to get mail forwarded through mailbox that does not this 
kind of rewriting, should configure the forwarder as trusted/internal for 
this case.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Chernobyl was an Windows 95 beta test site.

Re: Blacklisting By Mail Server Rather Than By An Email Address

[johnjinsf]

RW-15 wrote:
> 
> 
> I don't think there is a way to blacklist a server unless the provider
> allows you to create SA rules. 
> 

Many thanks for your replies and suggestions.

I haven't seen where my hoster allows for users to create rules,
but I'll open a ticket with their help desk to ask if they do.
-- 
View this message in context: 
http://old.nabble.com/Blacklisting-By-Mail-Server-Rather-Than-By-An-Email-Address-tp32622830p32627953.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Blacklisting based on SPF

[David F. Skoll]

On Mon, 10 Oct 2011 07:00:48 -0700
Marc Perkel  wrote:

[Blocking SPF "fail" mail]

> All forwarded email would fail SPF testing.  You would be blocking
> all hosted spam filtering services for example.

Nonsense.  If someone uses a hosted spam filtering servic for inbound mail,
then that person should turn off SPF checking on the back-end completely;
checking SPF and applying policy is the job of the hosted spam filter.
(If you're using a hosted anti-spam service that does *not* allow you
to apply fine-grained SPF policies, then it's time to switch.)

If someone uses a hosted filtering service for outbound mail, then
he/she just needs to publish appropriate SPF records listing the service's
egress IP addresses.

Regards,

David.

Re: Blacklisting based on SPF

[Daniel McDonald]

On 10/10/11 9:00 AM, "Marc Perkel"  wrote:

> 
> 
> On 10/7/2011 12:50 AM, Benny Pedersen wrote:
>> On 7 Oct 2011 00:28:49 -, John Levine wrote:
>>> Nobody with any interest in delivering the mail that their users want.
>>> The error rate is much, much too high.
>> 
>> how ?
>> 
> 
> All forwarded email would fail SPF testing.  You would be blocking all
> hosted spam filtering services for example.

"then you aren't doing it right".

If the hosted filtering is egress, then the address ranges of your egress
filter provider should be in your SPF statement.

If the hosted filtering is ingress, then the address ranges of your ingress
filter provider should be in your trusted-networks, so that spf will look at
the last-untrusted address for the source.

Mail-lists running on sane software will change the envelope address, so
there is no problem there.

So, what other bizarre corner cases are you talking about that break SPF?


-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281

Re: Blacklisting based on SPF

[Benny Pedersen]

On Mon, 10 Oct 2011 07:00:48 -0700, Marc Perkel wrote:

All forwarded email would fail SPF testing.  You would be blocking
all hosted spam filtering services for example.


this is easy to solve in spf or add the forwarding mta sender ip to 
spamassassin trusted_networks, reject msg ALWAYS says this to sender 
that are being rejected, fail is not a spf fault, i still not needing 
forwarded emails at all and i know how to do this from mail host i need 
forward from, if spf i so damm hard to use correct then use dkim :)

Re: Blacklisting based on SPF

[Marc Perkel]

On 10/7/2011 12:50 AM, Benny Pedersen wrote:

On 7 Oct 2011 00:28:49 -, John Levine wrote:

Nobody with any interest in delivering the mail that their users want.
The error rate is much, much too high.


how ?



All forwarded email would fail SPF testing.  You would be blocking all 
hosted spam filtering services for example.


--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: Blacklisting By Mail Server Rather Than By An Email Address

[RW]

On Mon, 10 Oct 2011 13:47:28 +0100
RW wrote:

> On Mon, 10 Oct 2011 03:47:27 -0700 (PDT)
> johnjinsf wrote:
> 
> > Is there a way of blacklisting the mail server which would prevent
> > any mail originating from that server being received?
> > 
> 
> I don't think there is a way to blacklist a server unless the provider
> allows you to create SA rules. If it does then:
> 
> 
> header  BAD_SERVER   X-Spam-Relays-Untrusted =~ /helo=mail.XXX.com /i
> score   BAD_SERVER   3.0

Forgot to escape the dots

/helo=mail\.XXX\.com/

Re: Blacklisting By Mail Server Rather Than By An Email Address

[RW]

On Mon, 10 Oct 2011 03:47:27 -0700 (PDT)
johnjinsf wrote:

> Is there a way of blacklisting the mail server which would prevent
> any mail originating from that server being received?
> 

I don't think there is a way to blacklist a server unless the provider
allows you to create SA rules. If it does then:


header  BAD_SERVER   X-Spam-Relays-Untrusted =~ /helo=mail.XXX.com /i
score   BAD_SERVER   3.0

Re: Blacklisting By Mail Server Rather Than By An Email Address

[Benny Pedersen]

On Mon, 10 Oct 2011 03:47:27 -0700 (PDT), johnjinsf wrote:

Is there a way of blacklisting the mail server which would prevent 
any mail

originating from that server being received?


is sender domain(s) rfc-ignorant ?, "sendmail -bv ab...@example.org" 
"sendmail -bv postmas...@example.org", what results come back to you ?, 
root is geting this mails :=)


if domain is rfc-ignorant please list them as so here 
http://www.rfc-ignorant.org/


next step depends on meta rules in spamassassin

"i got a new email address" was something i seen last year, but not 
much anymore :=)


suggested plugins to add SAGREY, with caches first time senders

Blacklisting By Mail Server Rather Than By An Email Address

[johnjinsf]

I have recently changed the company that hosts my email and they use
SpamAssassin.

In the SpamAssassin Configuration I have entered several email addresses in
the Blacklist which has worked fine.

One thing I have noticed with one particular spammer is that they send out
their emails using fake sender addresses
which bear no resemblance to either their domain (XXX.com) or their mail
server address (mail.XXX.com).

When an email is received in this form it is of no use to enter the fake
sender email address into the blacklist
because they use something very different the next time.

Is there a way of blacklisting the mail server which would prevent any mail
originating from that server being received?

Many thanks
-- 
View this message in context: 
http://old.nabble.com/Blacklisting-By-Mail-Server-Rather-Than-By-An-Email-Address-tp32622830p32622830.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Blacklisting based on SPF

[Dave Warren]

On 10/7/2011 12:17 PM, RW wrote:

On Fri, 07 Oct 2011 20:39:24 +0200
Robert Schetterer wrote:


in my case
there is so less left, passing postscreen, rbls, greylisting,
clamav-milter with sanesecurity and few other smtp checks, that nearly
null i.e
faked paypal mail getting at last to spamassassin where its stopped
mostly by other rules and rejected by spamass-milter, so using spf
check isnt hardly needed anymore,

His point was that SPF isn't there to catch spam, it there to identify
legitimate mail  from selected domains, and prevent it being falsely
identified as spam.


That's pretty much it.  I don't look at it as a spam blocking measure at 
all, but rather, it's utility is to avoid whitelisting forged mail.


Prior to SPF, I was apprehensive about whitelisting anything by domain 
since domains can be trivially forged, especially if it's a well-known 
domain (the domain of a household named company).  By only applying 
whitelist entries to mail that has a SPF or DKIM pass, I can whitelist 
by sender address/domain indiscriminately without fear that a spammer 
can take advantage of @paypal.com whitelists.


To me, false positives are a lot more important than filter misses.  
Users will tolerate a bit of spam, but blocking even a single legitimate 
message is unacceptable (yes, it's a real world risk, but it's still the 
goal), so being able to whitelist safely (completely, or just with a 
score) is critical.


--
Dave Warren, CEO
Hire A Hit Consulting Services
http://ca.linkedin.com/in/davejwarren

Suppressing backscatter (was Re: Blacklisting based on SPF)

[David F. Skoll]

On Fri, 07 Oct 2011 20:47:48 +0100
Martin Gregorie  wrote:

> And, at least for me, its been good for suppressing backscatter: since
> I've had a good SPF record I've has almost none.

Really??  You are very lucky.  We have an SPF record with a "-all"
clause and still get backscatter.  I believe that so few SMTP servers
validate SPF that the amount of backscatter it actually reduces is tiny.

Regards,

David.

Re: Blacklisting based on SPF

[Martin Gregorie]

On Fri, 2011-10-07 at 20:17 +0100, RW wrote:
> On Fri, 07 Oct 2011 20:39:24 +0200
> Robert Schetterer wrote:
> 
> > in my case
> > there is so less left, passing postscreen, rbls, greylisting,
> > clamav-milter with sanesecurity and few other smtp checks, that nearly
> > null i.e
> > faked paypal mail getting at last to spamassassin where its stopped
> > mostly by other rules and rejected by spamass-milter, so using spf
> > check isnt hardly needed anymore,
> 
> His point was that SPF isn't there to catch spam, it there to identify
> legitimate mail  from selected domains, and prevent it being falsely
> identified as spam.
>
And, at least for me, its been good for suppressing backscatter: since
I've had a good SPF record I've has almost none. That is all I use it
for.

Martin

Re: Blacklisting based on SPF

[RW]

On Fri, 07 Oct 2011 20:39:24 +0200
Robert Schetterer wrote:

> in my case
> there is so less left, passing postscreen, rbls, greylisting,
> clamav-milter with sanesecurity and few other smtp checks, that nearly
> null i.e
> faked paypal mail getting at last to spamassassin where its stopped
> mostly by other rules and rejected by spamass-milter, so using spf
> check isnt hardly needed anymore,

His point was that SPF isn't there to catch spam, it there to identify
legitimate mail  from selected domains, and prevent it being falsely
identified as spam.

Re: Blacklisting based on SPF

[Robert Schetterer]

Am 07.10.2011 20:24, schrieb Dave Warren:
> On 10/7/2011 1:12 AM, Robert Schetterer wrote:
>> in my eyes the whole idea of spf was broken from beginning
>> but do what you want, no need for flame
>> in my real world it makes more problems then helping in antispam
>> i removed spf checks from my servers, in spamd its used with nearly no
>> points
>> there are better more effective ways to reject unwanted mails
>> but youre free, do it like you want, analyse your logs
>> then you will see, if it helps at your side
>> everbody has its own spam, there are less
>> universal recommands, antispam is daily work in analyse and reaction
> 
> The trick with SPF is to stop using it for rejecting mail, it doesn't do
> a good job at that.  

jep

It's not really a spam-fighting technique at all,
> as much as an identification technique.  What you do with that

jep

> identification is where it gets interesting; what it does do well is
> allow you to whitelist known-good (or at least wanted) senders, allowing
> you to exempt mail you know you want from expensive content filtering.
> 
> PayPay is a good example, love 'em or hate 'em, there's no point running
> mail from PayPal through any sort of content based spam filtering, and
> SPF can tell you that a message claiming to be from PayPal really is
> from PayPal (but it can't reliably tell you that a message *isn't* from
> PayPal, due to forwarding, possible DNS problems, possible SPF
> configuration errors, etc)

in my case
there is so less left, passing postscreen, rbls, greylisting,
clamav-milter with sanesecurity and few other smtp checks, that nearly
null i.e
faked paypal mail getting at last to spamassassin where its stopped
mostly by other rules and rejected by spamass-milter, so using spf check
isnt hardly needed anymore, until in most cases its useless
or does make trouble, but feel free using spf-checks as you want
it may help in some setups



> 
> 


-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: Blacklisting based on SPF

[Dave Warren]

On 10/7/2011 1:12 AM, Robert Schetterer wrote:

in my eyes the whole idea of spf was broken from beginning
but do what you want, no need for flame
in my real world it makes more problems then helping in antispam
i removed spf checks from my servers, in spamd its used with nearly no
points
there are better more effective ways to reject unwanted mails
but youre free, do it like you want, analyse your logs
then you will see, if it helps at your side
everbody has its own spam, there are less
universal recommands, antispam is daily work in analyse and reaction


The trick with SPF is to stop using it for rejecting mail, it doesn't do 
a good job at that.  It's not really a spam-fighting technique at all, 
as much as an identification technique.  What you do with that 
identification is where it gets interesting; what it does do well is 
allow you to whitelist known-good (or at least wanted) senders, allowing 
you to exempt mail you know you want from expensive content filtering.


PayPay is a good example, love 'em or hate 'em, there's no point running 
mail from PayPal through any sort of content based spam filtering, and 
SPF can tell you that a message claiming to be from PayPal really is 
from PayPal (but it can't reliably tell you that a message *isn't* from 
PayPal, due to forwarding, possible DNS problems, possible SPF 
configuration errors, etc)



--
Dave Warren, CEO
Hire A Hit Consulting Services
http://ca.linkedin.com/in/davejwarren

Re: Blacklisting based on SPF

[Ned Slider]

On 07/10/11 13:27, Daniel McDonald wrote:


Something like this Unverified Yahoo rule I shameless stole from Mark
Martinec:



I have some similar rules...


header __L_FROM_Y1   From:addr =~ m{[@.]yahoo\.com$}i
header __L_FROM_Y2   From:addr =~ m{\@yahoo\.com\.(ar|br|cn|hk|my|sg)$}i
header __L_FROM_Y3   From:addr =~ m{\@yahoo\.co\.(id|in|jp|nz|uk)$}i
header __L_FROM_Y4   From:addr =~
m{\@yahoo\.(ca|de|dk|es|fr|gr|ie|it|pl|se)$}i


and thought I'd share my updated list of Yahoo TLDs as you're missing a few:

header		__LOCAL_FROM_YAHOO1	From:addr =~ 
/\@yahoo\.com\.(ar|br|cn|hk|mx|my|ph|sg)$/i
header		__LOCAL_FROM_YAHOO2	From:addr =~ 
/\@yahoo\.co\.(id|in|jp|nz|th|uk)$/i
header		__LOCAL_FROM_YAHOO3	From:addr =~ 
/\@yahoo\.(ca|cn|de|dk|es|fr|gr|ie|in|it|pl|ru|se)$/i

Re: Blacklisting based on SPF

[Daniel McDonald]

On 10/7/11 3:49 AM, "Julian Yap"  wrote:

> On Thu, Oct 6, 2011 at 3:09 PM, David F. Skoll  
> wrote:
>> On 7 Oct 2011 00:28:49 -
>> "John Levine"  wrote:
>> 
 Does anyone blacklist based on SPF?
>> 
>>> Nobody with any interest in delivering the mail that their users want.
>>> The error rate is much, much too high.
>> 
>> It depends.  I very confidently blacklist mail from "roaringpenguin.com
>>  "
>> that fails to pass SPF.  That's my own domain, of course.
> 
> What do your rules look like for this scenario?
> 

Something like this Unverified Yahoo rule I shameless stole from Mark
Martinec:

header __L_ML1   Precedence =~ m{\b(list|bulk)\b}i
header __L_ML2   exists:List-Id
header __L_ML3   exists:List-Post
header __L_ML4   exists:Mailing-List
header __L_HAS_SNDR  exists:Sender
meta   __L_VIA_ML__L_ML1 || __L_ML2 || __L_ML3 || __L_ML4 ||
__L_HAS_SNDR
header __L_FROM_Y1   From:addr =~ m{[@.]yahoo\.com$}i
header __L_FROM_Y2   From:addr =~ m{\@yahoo\.com\.(ar|br|cn|hk|my|sg)$}i
header __L_FROM_Y3   From:addr =~ m{\@yahoo\.co\.(id|in|jp|nz|uk)$}i
header __L_FROM_Y4   From:addr =~
m{\@yahoo\.(ca|de|dk|es|fr|gr|ie|it|pl|se)$}i
meta   __L_FROM_YAHOO __L_FROM_Y1 || __L_FROM_Y2 || __L_FROM_Y3 ||
__L_FROM_Y4
header __L_FROM_GMAIL From:addr =~ m{\@gmail\.com$}i
meta L_UNVERIFIED_YAHOO  !DKIM_VALID && !DKIM_VALID_AU && __L_FROM_YAHOO
&& !__L_VIA_ML
priority L_UNVERIFIED_YAHOO  500
scoreL_UNVERIFIED_YAHOO  2.5
meta L_UNVERIFIED_GMAIL  !DKIM_VALID && !DKIM_VALID_AU && __L_FROM_GMAIL
&& !__L_VIA_ML
priority L_UNVERIFIED_GMAIL  500
scoreL_UNVERIFIED_GMAIL  2.5



It would be nice to have a construct like "blacklist_unless_spf" or
"blacklist_unless_auth"  that did all of this for me...


-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281

Re: Blacklisting based on SPF

[David F. Skoll]

On Thu, 6 Oct 2011 22:49:47 -1000
Julian Yap  wrote:

> What do your rules look like for this scenario? [blocking for SPF
> fail for select domains.]

Ah, well.  We don't implement those policies with SpamAssassin, so I can't
post anything useful.

Regards,

David.

Re: Blacklisting based on SPF

[Benny Pedersen]

On Thu, 6 Oct 2011 22:49:47 -1000, Julian Yap wrote:

What do your rules look like for this scenario?


blacklist_from *@example.org
whitelist_from_spf *@example.org

adjust so blacklist score will be neotral for spf pass users

dont use *@example.org if you need to have strict whitelist of specific 
sender


so if spf fails it will be added blacklist_from score, if spf pass its 
neotral score

Re: Blacklisting based on SPF

[Julian Yap]

On Thu, Oct 6, 2011 at 3:09 PM, David F. Skoll wrote:

> On 7 Oct 2011 00:28:49 -
> "John Levine"  wrote:
>
> > >Does anyone blacklist based on SPF?
>
> > Nobody with any interest in delivering the mail that their users want.
> > The error rate is much, much too high.
>
> It depends.  I very confidently blacklist mail from "roaringpenguin.com"
> that fails to pass SPF.  That's my own domain, of course.
>
> With somewhat less (but still pretty high) confidence, I block mail
> from paypal.com and ebay.com if it fails SPF (including "softfail")
>
> SPF is most effective when used judiciously for specific domains.  It's
> pretty useless to make blanket SPF rules that cover unknown domains.
>
>
What do your rules look like for this scenario?

Julian

Re: Blacklisting based on SPF

[Robert Schetterer]

Am 07.10.2011 10:03, schrieb Benny Pedersen:
> On Fri, 07 Oct 2011 09:54:09 +0200, Robert Schetterer wrote:
>> but wouldnt recommend it anyway
> 
> why would i like to whitelist a unknown spammer ?
> 
> thinking more about it would get me mad :-)
> 
> 

in my eyes the whole idea of spf was broken from beginning
but do what you want, no need for flame
in my real world it makes more problems then helping in antispam
i removed spf checks from my servers, in spamd its used with nearly no
points
there are better more effective ways to reject unwanted mails
but youre free, do it like you want, analyse your logs
then you will see, if it helps at your side
everbody has its own spam, there are less
universal recommands, antispam is daily work in analyse and reaction
-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: Blacklisting based on SPF

[Benny Pedersen]

On Fri, 07 Oct 2011 09:54:09 +0200, Robert Schetterer wrote:

but wouldnt recommend it anyway


why would i like to whitelist a unknown spammer ?

thinking more about it would get me mad :-)

Re: Blacklisting based on SPF

[Benny Pedersen]

On Thu, 6 Oct 2011 21:09:59 -0400, David F. Skoll wrote:

SPF is most effective when used judiciously for specific domains.  
It's

pretty useless to make blanket SPF rules that cover unknown domains.


whitelist_from_spf rules ? :-)

my rule of thump is:

def_whitelist_from_spf *@example.org
whitelist_from_spf u...@example.net

so give more negstive scores to more restricted spf pass

Re: Blacklisting based on SPF

[Robert Schetterer]

Am 07.10.2011 09:50, schrieb Benny Pedersen:
> On 7 Oct 2011 00:28:49 -, John Levine wrote:
>> Nobody with any interest in delivering the mail that their users want.
>> The error rate is much, much too high.
> 
> how ?
> 
> 

good spammers , usally have valid spf dns entries
so if you want blacklist with spf do it selective
i.e with some milter
but wouldnt recommend it anyway
-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: Blacklisting based on SPF

[Benny Pedersen]

On 7 Oct 2011 00:28:49 -, John Levine wrote:
Nobody with any interest in delivering the mail that their users 
want.

The error rate is much, much too high.


how ?

Re: Blacklisting based on SPF

[David F. Skoll]

On 7 Oct 2011 00:28:49 -
"John Levine"  wrote:

> >Does anyone blacklist based on SPF?

> Nobody with any interest in delivering the mail that their users want.
> The error rate is much, much too high.

It depends.  I very confidently blacklist mail from "roaringpenguin.com"
that fails to pass SPF.  That's my own domain, of course.

With somewhat less (but still pretty high) confidence, I block mail
from paypal.com and ebay.com if it fails SPF (including "softfail")

SPF is most effective when used judiciously for specific domains.  It's
pretty useless to make blanket SPF rules that cover unknown domains.

Regards,

David.

Re: Blacklisting based on SPF

[John Levine]

In article 
 you write:
>-=-=-=-=-=-
>
>I've noticed some trojans with addresses from usps.com slip through.
>
>Does anyone blacklist based on SPF?

Nobody with any interest in delivering the mail that their users want.
The error rate is much, much too high.

R's,
John

Re: Blacklisting based on SPF

[Benny Pedersen]

On Wed, 5 Oct 2011 11:01:12 -1000, Julian Yap wrote:

Ive noticed some trojans with addresses from usps.com [1] slip
through.


ups.com ?


Does anyone blacklist based on SPF?


not needed since all spf domains is blacklisted, and scored neotral in 
spamassassin, until you use whitelist_from_spf or def_whitelist_from_spf 
sender email, and it will only gives neative score if its passing


also remember From: is not envelope sender, does spf use that header in 
your test ?


if it does then your spf test is brokken

have you set envelope_sender_header in local.cf ?

perldoc Mail::SpamAssassin::Conf

I took a look at the source for SpamAssassin/Plugin/SPF.pm but it 
only

has evaluation rules for whitelisting:
   $self->register_eval_rule ("check_for_spf_whitelist_from");
  $self->register_eval_rule ("check_for_def_spf_whitelist_from");


its not needed to have blacklist

Re: Blacklisting based on SPF

[Michael Scheidell]

On 10/5/11 5:01 PM, Julian Yap wrote:
I've noticed some trojans with addresses from usps.com 
 slip through.


Does anyone blacklist based on SPF?

I took a look at the source for SpamAssassin/Plugin/SPF.pm but it only 
has evaluation rules for whitelisting:

  $self->register_eval_rule ("check_for_spf_whitelist_from");
  $self->register_eval_rule ("check_for_def_spf_whitelist_from");

Thanks,
Julian

I tried blacklist_from *@usps.com with an whitelist_from.  (would even 
themselves out...)
problem is.. if I send to xmail, and xmail fwds (incorrectly), OR, dns 
doesn't answer in time, you lose email.


best to write a metarule.  put your def_ whitelist from (7 points), and 
set up some metarules.




--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation

   * Best Mobile Solutions Product of 2011
   * Best Intrusion Prevention Product
   * Hot Company Finalist 2011
   * Best Email Security Product
   * Certified SNORT Integrator

__
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com/
__  
 

Blacklisting based on SPF

[Julian Yap]

I've noticed some trojans with addresses from usps.com slip through.

Does anyone blacklist based on SPF?

I took a look at the source for SpamAssassin/Plugin/SPF.pm but it only has
evaluation rules for whitelisting:
  $self->register_eval_rule ("check_for_spf_whitelist_from");
  $self->register_eval_rule ("check_for_def_spf_whitelist_from");

Thanks,
Julian

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[David F. Skoll]

On Tue, 08 Feb 2011 17:04:37 +
Steve Freegard  wrote:

> Sure - credit where it is due; I've you to the 'Thanks' section.

Thanks.  And also, my apologies for posting to the list... that was supposed
to be a private message. :(

/me mutters something about email amateurs not understanding how email works...

Regards,

David.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Steve Freegard]

Hi David,

On 08/02/11 15:57, David F. Skoll wrote:

Hi, Steve,


http://www.fsl.com/index.php/resources/whitepapers/99


Interesting.  I think you should credit me for this:

"Once that has been proven then that â is exempted from further
greylisting for 40 days since it was last seen."

Our CanIt system has been doing that since at least 2005, and AFAIK
was the first to do so.  I understand if you don't want to credit a direct
competitor :), but at least include my name.


I wasn't aware and have no way of verifying that.  Besides it's a pretty 
obvious thing to do when you look at what is trying to be proven.  We've 
been doing this for ages too (since v1 in 2007).



Also, I mentioned greylisting before Evan Harris's paper.  It's here:

http://groups.google.com/group/comp.mail.sendmail/msg/ef7624049e1193a2?hl=en



Sure - credit where it is due; I've you to the 'Thanks' section.

Cheers,
Steve.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[David F. Skoll]

On Tue, 08 Feb 2011 15:47:12 +
Steve Freegard  wrote:

> See http://www.fsl.com/index.php/resources/whitepapers/99

"Once that has been proven then that 'hostid' is exempted from further
greylisting for 40 days since it was last seen."

:)  Our CanIt system has been doing this since at least 2005.

Also, I wrote about greylisting a few months before Evan Harris
published his paper:

http://groups.google.com/group/comp.mail.sendmail/msg/ef7624049e1193a2?hl=en

Regards,

David.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[David F. Skoll]

Hi, Steve,

> http://www.fsl.com/index.php/resources/whitepapers/99

Interesting.  I think you should credit me for this:

"Once that has been proven then that â is exempted from further
greylisting for 40 days since it was last seen."

Our CanIt system has been doing that since at least 2005, and AFAIK
was the first to do so.  I understand if you don't want to credit a direct
competitor :), but at least include my name.

Also, I mentioned greylisting before Evan Harris's paper.  It's here:

http://groups.google.com/group/comp.mail.sendmail/msg/ef7624049e1193a2?hl=en

Regards,

David.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Steve Freegard]

On 19/01/11 15:02, David F. Skoll wrote:

On Wed, 19 Jan 2011 09:56:47 -0500
Lee Dilkie  wrote:


The second was that I've found that the other spam-catching filtering
is doing a much better job than it was years ago and turning off
greylisting didn't adversely affect the amount of spam that got
through.


That's possibly true, but look at this.

A greylisted message: mimedefang[17175]: p0I4xvRE017628: Filter time is 85ms
A scanned message:mimedefang[17175]: p0I50ACP017683: Filter time is 906ms

On a busy system, this can make a huge difference.  SpamAssassin scanning
is by no means cheap.



I know this thread is a bit old now; but at the time this was being 
discussed I was running a test as I decided to revisit greylisting and 
see if it was worth keeping in our products.


I found the results very interesting (to me at least), so I decided to 
write a whitepaper and share my results:


See http://www.fsl.com/index.php/resources/whitepapers/99

Kind regards,
Steve.

Re: Fwd: Re: Q about short-circuit over ruling blacklisting rule

[Nels Lindquist]

On 2011/01/18 9:49 AM, J4 wrote:

> This is pretty much what I would like to achieve, & the reason I
> decided not to use Dovecot Sieve (apart from me being incapable of
> setting it.  ;)  ).  
> 
> Parse the SPAM during the SMPT session and use only RAM: Perfect.
>  
> I would still like to notify the connecting SMTP client with a reject
> message.  Real spammers are uninterested anyway, but legitimate
> e-mailers would be, although this is not essential to let them know.
> 
> The problem is that I don't know how to achieve this with postfix :( 
> The postfix set-up I have is below (master.cf), but I do not know for
> certain that it is filtering during the SMTP session afore it hits the
> disc, and I have not found any information about how to configure this. 
> My hunt for guides goes on.

Since you've already implemented a milter solution, I'd like to mention
for posterity's sake that when using Postfix there is another possible
solution, using spampd as a before-queue content filter to reject
messages at arbitrary spamassassin classification thresholds.

It's even in the wiki:
http://wiki.apache.org/spamassassin/IntegratePostfixViaSpampd

Nels Lindquist

Re: Suspicious URL:Re: Suspicious URL:Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Daniel McDonald]

On 1/19/11 2:35 PM, "John Hardin"  wrote:

> On Wed, 19 Jan 2011, Daniel McDonald wrote:
> 
>> On 1/19/11 10:17 AM, "John Hardin"  wrote:
>> 
>>> On Wed, 19 Jan 2011, Lee Dilkie wrote:
>>> 
 Don't get me wrong, I liked GL but there are a number of big ISPs that
 have quite long retry timeouts (for some reason, sympatico comes to
 mind) and it got to be too annoying.
>>> 
>>> ...and when you encounter a big ISP that does this, do you notify their
>>> postmaster so they can fix the problem?
>> 
>> Or add a grey-listing exception and publish it to the sqlgrey list so that
>> the rest of us can also add an exception?
> 
> Is the whitelist available standalone for those of us who don't use
> sqlgrey? I couldn't see it and didn't want to grab the entire tarball.
> 
> (As I was researching this I came across a posting to the sqlgrey list
> from 2005 mentioning a whitelist entry request on behalf of a C/R vendor,
> and my first thought was "what, we want to _encourage_ C/R?")
The files are accessible at
http://sqlgrey.bouton.name

The available files are MD5SUMS, README, clients_fqdn_whitelist,
clients_ip_whitelist, dyn_fqdn.regexp, smtp_server.regexp

There is a script in the tarball to retrieve the changed files by comparing
the published md5sum with that on disk and only pulling down those that are
different.


-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281

Re: Suspicious URL:Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[John Hardin]

On Wed, 19 Jan 2011, Daniel McDonald wrote:


On 1/19/11 10:17 AM, "John Hardin"  wrote:


On Wed, 19 Jan 2011, Lee Dilkie wrote:


Don't get me wrong, I liked GL but there are a number of big ISPs that
have quite long retry timeouts (for some reason, sympatico comes to
mind) and it got to be too annoying.


...and when you encounter a big ISP that does this, do you notify their
postmaster so they can fix the problem?


Or add a grey-listing exception and publish it to the sqlgrey list so that
the rest of us can also add an exception?


Is the whitelist available standalone for those of us who don't use 
sqlgrey? I couldn't see it and didn't want to grab the entire tarball.


(As I was researching this I came across a posting to the sqlgrey list 
from 2005 mentioning a whitelist entry request on behalf of a C/R vendor, 
and my first thought was "what, we want to _encourage_ C/R?")


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  One difference between a liberal and a pickpocket is that if you
  demand your money back from a pickpocket he will not question your
  motives.  -- William Rusher
---
 4 days until John Moses Browning's 156th Birthday

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Ted Mittelstaedt]

On 1/19/2011 8:06 AM, Lee Dilkie wrote:


On 1/19/2011 10:02 AM, David F. Skoll wrote:

On Wed, 19 Jan 2011 09:56:47 -0500
Lee Dilkie  wrote:


The second was that I've found that the other spam-catching filtering
is doing a much better job than it was years ago and turning off
greylisting didn't adversely affect the amount of spam that got
through.

That's possibly true, but look at this.

A greylisted message: mimedefang[17175]: p0I4xvRE017628: Filter time is 85ms
A scanned message:mimedefang[17175]: p0I50ACP017683: Filter time is 906ms

On a busy system, this can make a huge difference.  SpamAssassin scanning
is by no means cheap.

Regards,

David.


Agreed there, I did have to install the compiled regex package to get SA
speeds up enough to handle the increased load (my server is not even
close to yours in performance but I did drop SA time from 10-30s to <3s).

Don't get me wrong, I liked GL but there are a number of big ISPs that
have quite long retry timeouts (for some reason, sympatico comes to
mind) and it got to be too annoying.



In our experience the large ISPs don't have long retry timeouts.  What 
they have are multiple outbound mailservers.  They will try from 1 
server and when they get the error 4xx they shift the outbound message 
to another server.  This happens until all of their outbound servers

have been greylisted for the one message, then it goes through.

In my opinion (we use greylist-milter) the greylist developers are
basically their own worst enemies here.  They distribute a list of known
ISPs that round-robin outbound mail.  But the list is very old and
isn't a quarter of the ones that actually do it.

So an admin inexperienced with greylisting installs it and gets the 
experience your relating and then assumes that greylisting is no good.


Note that I am not assuming your inexperienced or that you don't already
know all about this problem.  You just didn't mention it so I didn't
want others who might come across this posting who might not be 
experienced with this to not know about it.


In our case greylisting is very cheap on CPU cycles.  But the regex 
matching and virus filtering is quite expensive.  And worse, we have

to pass everything to the users including the spam that we have tagged,
so we cannot do the logical thing and put SA first and then just throw
away from further processing everything that is determined to be spam.
Instead we have to put the virus filtering first (because we are allowed
to toss virus-infected mail) which means everything gets both scanned
for spam (except viruses) and virus filtered.

So we prefilter with greylist-milter and it really does indeed 
tremendously reduce the load on the server.  But you really do have to

explain to your users what is going on for it to work, and you
have to thoroughly investigate every mail complaint to make sure that
it's not caused by a round-robin ISP that you don't have in your
exception list.  And you have be alert for hosts like
craigslist.org because those bastards
fail mail delivery EVEN IF they get an error 4xx in violation of
the RFCs.

Ted


who knows, all the code is still there and I might switch it on again in
the future.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Matt]

 The legitimate mail that passes through my mail server comes from
 hosts / networks I might not hear from again for months, by which
 time I have to potentially wait 24 hours for the greylisting / mail
 server to try again.
>>
>> I run greylisting on an email server with several thousand email
>> accounts and think its great.  Reduced system load drastically.  I
>> also have a perl script I have wrote that runs every minute and looks
>> at all messages that arrived in last 60 seconds and if spamassassin
>> gave them a score of less then 5 it adds the sending MTA to a
>> whitelist.  It also removes any from the whitelist that have sent a
>> message that scored over 5.  The whitelist goes back 6 months and is
>> continually refreshed by the script.  The whitelist typically has 40K
>> IP's in it.
>>
>> As a result no one even notices the greylisting, AFAIK...
>
> Is that the greylist whitelist or the SA whitelist?

It whitelists those IP's from being greylisted in future.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Ted Mittelstaedt]

On 1/19/2011 9:25 AM, Matt wrote:

The legitimate mail that passes through my mail server comes from
hosts / networks I might not hear from again for months, by which
time I have to potentially wait 24 hours for the greylisting / mail
server to try again.


I run greylisting on an email server with several thousand email
accounts and think its great.  Reduced system load drastically.  I
also have a perl script I have wrote that runs every minute and looks
at all messages that arrived in last 60 seconds and if spamassassin
gave them a score of less then 5 it adds the sending MTA to a
whitelist.  It also removes any from the whitelist that have sent a
message that scored over 5.  The whitelist goes back 6 months and is
continually refreshed by the script.  The whitelist typically has 40K
IP's in it.

As a result no one even notices the greylisting, AFAIK...


Is that the greylist whitelist or the SA whitelist?

Ted

Re: Suspicious URL:Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Henrik K]

On Wed, Jan 19, 2011 at 11:14:29AM -0600, Daniel McDonald wrote:
> On 1/19/11 10:17 AM, "John Hardin"  wrote:
> 
> > On Wed, 19 Jan 2011, Lee Dilkie wrote:
> > 
> >> Don't get me wrong, I liked GL but there are a number of big ISPs that
> >> have quite long retry timeouts (for some reason, sympatico comes to
> >> mind) and it got to be too annoying.
> > 
> > ...and when you encounter a big ISP that does this, do you notify their
> > postmaster so they can fix the problem?
> 
> Or add a grey-listing exception and publish it to the sqlgrey list so that
> the rest of us can also add an exception?

Or use DNSWL and co which include many major relays?

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Matt]

>> The legitimate mail that passes through my mail server comes from
>> hosts / networks I might not hear from again for months, by which
>> time I have to potentially wait 24 hours for the greylisting / mail
>> server to try again.

I run greylisting on an email server with several thousand email
accounts and think its great.  Reduced system load drastically.  I
also have a perl script I have wrote that runs every minute and looks
at all messages that arrived in last 60 seconds and if spamassassin
gave them a score of less then 5 it adds the sending MTA to a
whitelist.  It also removes any from the whitelist that have sent a
message that scored over 5.  The whitelist goes back 6 months and is
continually refreshed by the script.  The whitelist typically has 40K
IP's in it.

As a result no one even notices the greylisting, AFAIK...

Re: Suspicious URL:Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Daniel McDonald]

On 1/19/11 10:17 AM, "John Hardin"  wrote:

> On Wed, 19 Jan 2011, Lee Dilkie wrote:
> 
>> Don't get me wrong, I liked GL but there are a number of big ISPs that
>> have quite long retry timeouts (for some reason, sympatico comes to
>> mind) and it got to be too annoying.
> 
> ...and when you encounter a big ISP that does this, do you notify their
> postmaster so they can fix the problem?

Or add a grey-listing exception and publish it to the sqlgrey list so that
the rest of us can also add an exception?

I seldom have problems with large mailers.  Most of my greylisting issues
come from small organizations.  I usually end up exempting them from
grey-listing, after we get their DNS cleaned up


-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[John Hardin]

On Wed, 19 Jan 2011, Lee Dilkie wrote:

Don't get me wrong, I liked GL but there are a number of big ISPs that 
have quite long retry timeouts (for some reason, sympatico comes to 
mind) and it got to be too annoying.


...and when you encounter a big ISP that does this, do you notify their 
postmaster so they can fix the problem?


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Taking my gun away because I *might* shoot someone is like cutting
  my tongue out because I *might* yell "Fire!" in a crowded theater.
  -- Peter Venetoklis
---
 4 days until John Moses Browning's 156th Birthday

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Lee Dilkie]

On 1/19/2011 10:02 AM, David F. Skoll wrote:
> On Wed, 19 Jan 2011 09:56:47 -0500
> Lee Dilkie  wrote:
>
>> The second was that I've found that the other spam-catching filtering
>> is doing a much better job than it was years ago and turning off
>> greylisting didn't adversely affect the amount of spam that got
>> through.
> That's possibly true, but look at this.
>
> A greylisted message: mimedefang[17175]: p0I4xvRE017628: Filter time is 85ms
> A scanned message:mimedefang[17175]: p0I50ACP017683: Filter time is 906ms
>
> On a busy system, this can make a huge difference.  SpamAssassin scanning
> is by no means cheap.
>
> Regards,
>
> David.

Agreed there, I did have to install the compiled regex package to get SA
speeds up enough to handle the increased load (my server is not even
close to yours in performance but I did drop SA time from 10-30s to <3s).

Don't get me wrong, I liked GL but there are a number of big ISPs that
have quite long retry timeouts (for some reason, sympatico comes to
mind) and it got to be too annoying.

who knows, all the code is still there and I might switch it on again in
the future.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[David F. Skoll]

On Wed, 19 Jan 2011 09:56:47 -0500
Lee Dilkie  wrote:

> The second was that I've found that the other spam-catching filtering
> is doing a much better job than it was years ago and turning off
> greylisting didn't adversely affect the amount of spam that got
> through.

That's possibly true, but look at this.

A greylisted message: mimedefang[17175]: p0I4xvRE017628: Filter time is 85ms
A scanned message:mimedefang[17175]: p0I50ACP017683: Filter time is 906ms

On a busy system, this can make a huge difference.  SpamAssassin scanning
is by no means cheap.

Regards,

David.

Re: Greylisting delay (was Re: Q about short-circuit over ruling blacklisting rule)

[Lee Dilkie]

I recently gave up on greylisting after using it for years as well.

Two reasons really, one was the complaints from users (and I found that
they often asked folks to "send mail to me twice" to try and get mail to
"work better" and that was just embarrassing).

The second was that I've found that the other spam-catching filtering is
doing a much better job than it was years ago and turning off
greylisting didn't adversely affect the amount of spam that got through.

-lee


On 1/18/2011 5:41 PM, Warren Togami Jr. wrote:
> On 01/18/2011 12:31 PM, David F. Skoll wrote:
>> On Tue, 18 Jan 2011 22:18:20 +
>> Gary Forrest  wrote:
>>
>>> Interesting 2 of our 3 scanning heads use a grey list system that
>>> uses /32 addresses as part of the process, these two servers have
>>> 100's of emails delayed for well over a day. Our 3rd scanning head
>>> uses a grey list system that is less granular /24 ,  this does not.
>>
>> Ah, I should mention that we use a /24 for greylisting for IPv4 and a
>> /64 for IPv6.  On the other hand, we also add a hash of the subject
>> into the greylisting tuple so it becomes:
>
> I recently gave up entirely on greylisting after:
>
> * Last week I discovered /24 was not good enough for redelivery
> attempts at one major ISP.  All mail from that ISP was failing for the
> past month except in rare cases where randomly the same /24 attempted
> delivery within the time window.
>
> * Years of complaints of mail delivery delays or failures from my
> users.  They had began creating gmail accounts in order to bypass. 
> They kept running into too many cases of broken individual mail
> servers (major companies!) who failed to redeliver.
>
> Users don't care about "so and so is violating RFC-XXX".  They are
> trying to get business done and it was simply causing too many problems.
>
> Warren