Re: A Plan to Stop Violence on Social Media

[Rob McEwen]

On 12/15/2015 11:19 PM, Wrolf wrote:

Would it be practical to use the Spamassassin techniques of Bayesian
filtering and RBL lists to block ISIS on social media?


Bayes would have a high potential for false positives... such as 
blocking a news story about ISIS, or blocking a discussion about how to 
stop ISIS, or blocking an innocent discussion about the middle east, or 
an innocent discussion about Islam.


But ISIS isn't an nearly as much of an extremely fast moving and 
frequently morphing target as spam. Therefore, the techniques used don't 
have to have as much risk of FPs (since they wouldn't have to be as 
real-time-reacting to new scenarios) and should therefore be very 
surgically targeted towards 100% confirmed ISIS sources.


At the same time, beware of top-down solutions where governments impose 
free speech ("thought control") restrictions. Such powers could easily 
be abused in the future for nefarious purposes, such as suppressing 
criticism of the current party in power, etc.


This could be a "slippery slope".

--
Rob McEwen
+1 478-475-9032

Re: A Plan to Stop Violence on Social Media

[Marc Perkel]

Probably yes. But talk about opening a can of worms. If you can detect 
ISIS you can detect anything.


On 12/15/15 20:19, Wrolf wrote:

Stop me if you've heard this one.

Would it be practical to use the Spamassassin techniques of Bayesian 
filtering and RBL lists to block ISIS on social media?


Wrolf
wr...@wrolf.net 


--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

A Plan to Stop Violence on Social Media

[Wrolf]

Stop me if you've heard this one.

Would it be practical to use the Spamassassin techniques of Bayesian
filtering and RBL lists to block ISIS on social media?

Wrolf
wr...@wrolf.net

Re: feed spamassassin with a catch-all address

[Axb]

On 12/15/2015 11:33 PM, Kevin A. McGrail wrote:

On 12/15/2015 5:25 PM, Juerg Reimann wrote:

I have a domain which gets a lot of spam to non-existent addresses. I
thought why not set that domain to catch-all and feed all non-existent
addresses directly to spamassassin. Any thoughts why this could be a bad
idea? Of course any typos from real senders would also end up in sa,
however, I believe that's in this case negligible...

Thanks,
Juerg

This type of honeypot can find numerous bad actors and identify
dictionary attackers.  It has excellent merit and many people use this
type of data.  You might find it useful for blocking IPs, finding bad
URLs, identifying spam for bayes, etc.


easy to kill legit/ESP bulk and use the rest as bayes 
fodder...masschecks, etc, etc

Re: feed spamassassin with a catch-all address

[Martin Gregorie]

On Tue, 2015-12-15 at 23:25 +0100, Juerg Reimann wrote:
> I have a domain which gets a lot of spam to non-existent addresses. I
> thought why not set that domain to catch-all and feed all non
> -existent
> addresses directly to spamassassin. Any thoughts why this could be a
> bad
> idea? Of course any typos from real senders would also end up in sa,
> however, I believe that's in this case negligible...
> 
That's exactly what I do.

However, I also have:
- an ISP that runs grey-listing that I have enabled
- a private SA rule that adds points for unknown user names
- a quarantine system that holds spam for 7 days before 
  deleting it and that lists quarantined messages and stats
  as part of the daily logwatch report.

... so the very small amount of misclassified spam isn't lost.


Martin

Re: feed spamassassin with a catch-all address

[Kevin A. McGrail]

On 12/15/2015 5:25 PM, Juerg Reimann wrote:

I have a domain which gets a lot of spam to non-existent addresses. I
thought why not set that domain to catch-all and feed all non-existent
addresses directly to spamassassin. Any thoughts why this could be a bad
idea? Of course any typos from real senders would also end up in sa,
however, I believe that's in this case negligible...

Thanks,
Juerg
This type of honeypot can find numerous bad actors and identify 
dictionary attackers.  It has excellent merit and many people use this 
type of data.  You might find it useful for blocking IPs, finding bad 
URLs, identifying spam for bayes, etc.


Regards,
AKM

feed spamassassin with a catch-all address

[Juerg Reimann]

I have a domain which gets a lot of spam to non-existent addresses. I
thought why not set that domain to catch-all and feed all non-existent
addresses directly to spamassassin. Any thoughts why this could be a bad
idea? Of course any typos from real senders would also end up in sa,
however, I believe that's in this case negligible...

Thanks,
Juerg

Re: Trying Bayes / Redis

[Axb]

On 12/15/2015 10:57 PM, Marc Perkel wrote:

This Bayes Redis works GREAT. For years I've been trying to get bayes to
work and now finally IT WORKS


good news for once...
just watch memory usage... :)

Re: Trying Bayes / Redis

[Marc Perkel]

This Bayes Redis works GREAT. For years I've been trying to get bayes to 
work and now finally IT WORKS



--
Marc Perkel - Sales/Support
supp...@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400

Re: More on T_SPF_PERMERROR

[Bill Cole]

On 14 Dec 2015, at 21:42, Alex wrote:
[...]


I also don't think it's a DNS problem here, as it doesn't happen on
every message. There are also no other indications of problems with
DNS.


SPF records tend to push the limits of normal DNS, especially in record 
size, and can bring out edge case flaws in resolvers and firewalls that 
massage DNS which are otherwise hard to see.



Many times the domain actually has something wrong with SPF, but other
times openspf.org/why and kittermans say there's nothing wrong with
the domain.


Try https://dmarcian.com/spf-survey/ as well. It's very good at showing 
the details of SPF problems.


What you MAY be seeing are entirely transient problems. Complex SPF 
setups often come with redirects and includes that cross administrative 
boundaries and a variety of short TTLs. That combination can result in 
an SPF resolution that is "broken" only as a result of cached records 
that are expired a few minutes later and replaced by fresh corrected 
records.


Another source of persistent SPF errors is demonstrated in the 
cintas.com record. They've outsourced various aspects of their email ops 
to 3 different providers and put 4 includes of provider SPF records in 
their base record, one of which then re-includes one of the top-level 
includes. The result is the need to do 15 resolutions to expand that 
record: so it is out of spec and SHOULD return a PERMERROR. With that 
degree of outsourcing I doubt that there's anyone at Cintas who can even 
understand what a SPF record is, so the problem is almost certain to be 
persistent. That sort of problem is widespread in large organizations. 
If the last person who understood SPF makes the mistake of including 
outlook.com instead of (or in addition to) spf.protection.outlook.com on 
their way out the door, they effectively sabotage the ability to add 
another include for other purposes.



Other domains that fail, such as gmail.com and wellsfargo.com, report
softfail on kitterman when testing due to a redirect, apparently.
Would that be enough to cause this rule to legitimately report
temperror?


No. Softfail is a specifically defined non-error SPF result, caused by a 
"~all" default ("tail") component of the record. It is NOT an error, it 
is a chosen result.



Is there any way to easily retest the mails that have failed? I
suppose it's more difficult to test for T_SPF_HELO_TEMPERROR, but
every one of those seem to be legitimate failures anyway.


If you don't have a time machine and don't capture all of your DNS 
traffic, it is not possible to retest perfectly or know why either 
flavor of error occurred (although TEMPERROR is almost always the result 
of DNS timeouts.) However, there is the somewhat lame 'spfquery' script 
that is includes in the perl Mail::SPF module (which is what does SPF 
checking for SA) so you can always check locally rather than relying on 
a test on a distant website using different DNS servers.

Re: More on T_SPF_PERMERROR

[Joe Quinn]

On 12/15/2015 7:19 AM, Martin Gregorie wrote:

On Mon, 2015-12-14 at 21:42 -0500, Alex wrote:

Many times the domain actually has something wrong with SPF, but
other times openspf.org/why and kittermans say there's nothing wrong
with the domain.

Other domains that fail, such as gmail.com and wellsfargo.com, report
softfail on kitterman when testing due to a redirect,


For wellsfargo dmarcian.com/spf-survey follows the redirect link and
reports errors in that TXT (too many IPs and partial IPs). If
wellsfargo can make this mistake, so can others.

Could T_SPF_HELO_TEMPERROR be trying to say "I can't parse this
particular SPF TXT record"?


Martin
Syntax errors are permanent errors, as they will never go away on their 
own. I highly recommend reading 
http://www.rfc-editor.org/rfc/rfc7208.txt - it's a fairly easy read and 
important to understand well.


4.6.  Record Evaluation

   The check_host() function parses and interprets the SPF record to
   find a result for the current test.  The syntax of the record is
   validated first, and if there are any syntax errors anywhere in the
   record, check_host() returns immediately with the result "permerror",
   without further interpretation or evaluation.

Re: More on T_SPF_PERMERROR

[Martin Gregorie]

On Mon, 2015-12-14 at 21:42 -0500, Alex wrote:
> Many times the domain actually has something wrong with SPF, but
> other times openspf.org/why and kittermans say there's nothing wrong 
> with the domain.
> 
> Other domains that fail, such as gmail.com and wellsfargo.com, report
> softfail on kitterman when testing due to a redirect,
>
For wellsfargo dmarcian.com/spf-survey follows the redirect link and
reports errors in that TXT (too many IPs and partial IPs). If
wellsfargo can make this mistake, so can others.

Could T_SPF_HELO_TEMPERROR be trying to say "I can't parse this
particular SPF TXT record"?


Martin