Re: Running SA without the bayesian classifier

2014-08-12 Thread Matus UHLAR - fantomas

On 11.08.14 16:38, Matteo Dessalvi wrote:

I am planning to install SA on our SMTP MTAs, which deals only with
outgoing traffic generated in the internal network.
I am making the assumption that our clients are mostly sending 'clean'
email (I know, I am trusting *a lot* my users but nevertheless).

So the question is: how efficient will be SA without using the bayesian
classifier? Are all the remaining rulesets (apart from BAYES_*)
sufficient to shave off spam email?


It's gonna be very hard, but worth trying imho.
As already noted, most of RBL checks and ALL_TRUSTED have to be cleared out,
because their in first case useless, and the second would hit always - at
least it technically should, by definition.

That means, much of rules that push over limit will not hit. 
You still should not push required_score down, I remember outgoing mail

being blocked by inherited servers for hitting 7.0...

You can still use RBL checks like RCVD_IN_SORBS_*, RCVD_IN_XBL, URI BL's,
and razor/pyzor/dcc

However, I would try using BAYES, at least when you get some outbreaks.

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
My mind is like a steel trap - rusty and illegal in 37 states. 


Re: Running SA without the bayesian classifier

2014-08-12 Thread Matteo Dessalvi

Hi all.

Thanks for all the answers. I am afraid I was being naive.
I was explicitly thinking of a scenario like this: filter as
much as possible 'unsolicited email' sent by some (possibly)
'infected' account.

I thought that turning off the bayesian classifier (and the
RBL checks) would still let me able to catch the occasional
spam email. Of course there's already a ClamAV filtering
system for all the outgoing email.

In the past week one of our outgoing SMTP server was blacklisted
for 12 hours (just to be clear: it was not SpamHaus).
Unfortunately, looking at the logs did not give me any clues: there
were no spikes of bulk sending email to thousands of users or
anything particularly suspicious. And the black list manager did
not provide any additional information about the incident.

On 12.08.2014 08:43 Matus UHLAR wrote:

That means, much of rules that push over limit will not hit.

 You still should not push required_score down, I remember outgoing mail

being blocked by inherited servers for hitting 7.0...


I was thinking about using a 5.0 threshold but given your example
I guess I should push it up to 8.0.

On 11.08.2014 23:15, Karsten Bräckelmann wrote:

 Define spam.

 Running SA on your outgoing SMTP will not catch botnet generated junk,
 neither spam nor malware. This would require sniffing raw traffic. Or
 completely firewalling off outgoing port 25 connections.
 You explicitly mention your users (corporate or home?) sending mail.
 Are you talking about them possibly running bulk sending services, or
 hand crafted unsolicited mail to individual recipients?

If possible I would like to catch both but as already said this
gonna look quite hard. I will add Pyzor/DCC in the mix and see
if it can help.

On 11.08.2014 23:15, Karsten Bräckelmann wrote:

Unless there's a 419 gang operating from your internal network, there
might not be much left for SA with stock rules to classify spam...


No 'spam gang' so far but I will keep my eyes open :-).

Best regards,
  Matteo


Re: Running SA without the bayesian classifier

2014-08-12 Thread Matus UHLAR - fantomas

On 12.08.2014 08:43 Matus UHLAR wrote:

That means, much of rules that push over limit will not hit.
You still should not push required_score down, I remember outgoing mail
being blocked by inherited servers for hitting 7.0...


On 12.08.14 12:08, Matteo Dessalvi wrote:

I was thinking about using a 5.0 threshold but given your example
I guess I should push it up to 8.0.


if you use quarantine and watch the statistics, you can catch spam outbreaks
without unnecessary delaying mail. Note that your users might see manual
review as breaking their privacy...

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Support bacteria - they're the only culture some people have. 


Re: Running SA without the bayesian classifier

2014-08-12 Thread Karl Johnson
On Tue, Aug 12, 2014 at 6:08 AM, Matteo Dessalvi mte...@yahoo.it wrote:

 Hi all.

 Thanks for all the answers. I am afraid I was being naive.
 I was explicitly thinking of a scenario like this: filter as
 much as possible 'unsolicited email' sent by some (possibly)
 'infected' account.

 I thought that turning off the bayesian classifier (and the
 RBL checks) would still let me able to catch the occasional
 spam email. Of course there's already a ClamAV filtering
 system for all the outgoing email.

 In the past week one of our outgoing SMTP server was blacklisted
 for 12 hours (just to be clear: it was not SpamHaus).
 Unfortunately, looking at the logs did not give me any clues: there
 were no spikes of bulk sending email to thousands of users or
 anything particularly suspicious. And the black list manager did
 not provide any additional information about the incident.



I have the same kind of setup. I only scan outgoing email in case of a
compromised account being used to send spam. Last attack,
Amavis/Spamassassin blocked 83% of all outgoing spams ( 2390 passed out of
13938 ) so you can have some OK results even without using
bayes/RBL/SPF/DKIM checkup. DCC and URIBL help a lot. I still want/need to
go over 90%+ blocked.

Karl


Running SA without the bayesian classifier

2014-08-11 Thread Matteo Dessalvi

Hi all.

This may be a very stupid question but I would like to ask you all
anyway.

I am planning to install SA on our SMTP MTAs, which deals only with
outgoing traffic generated in the internal network.
I am making the assumption that our clients are mostly sending 'clean'
email (I know, I am trusting *a lot* my users but nevertheless).

So the question is: how efficient will be SA without using the bayesian
classifier? Are all the remaining rulesets (apart from BAYES_*)
sufficient to shave off spam email?

I am considering this scenario just because it will make the deployment
a little be easier, since I would not need a centralized Redis or MySQL
instance to keep the bayes data in a centralized way.

Thanks in advance.

Best regards,
   Matteo


Re: Running SA without the bayesian classifier

2014-08-11 Thread Kevin A. McGrail

On 8/11/2014 10:38 AM, Matteo Dessalvi wrote:

Hi all.

This may be a very stupid question but I would like to ask you all
anyway.

I am planning to install SA on our SMTP MTAs, which deals only with
outgoing traffic generated in the internal network.
I am making the assumption that our clients are mostly sending 'clean'
email (I know, I am trusting *a lot* my users but nevertheless).

So the question is: how efficient will be SA without using the bayesian
classifier? Are all the remaining rulesets (apart from BAYES_*)
sufficient to shave off spam email?
For a variety of reasons, we do not use bayesian classifier though the 
Redis backend has changed the primary concern.


But that aside, we are able to get extremely accurate filtering without 
Bayes and you can always work to bolt it on later.


Regards,
KAM


Re: Running SA without the bayesian classifier

2014-08-11 Thread Karsten Bräckelmann
On Mon, 2014-08-11 at 16:38 +0200, Matteo Dessalvi wrote:
 I am planning to install SA on our SMTP MTAs, which deals only with
 outgoing traffic generated in the internal network.

Outgoing traffic. That means, most DNSBLs are either completely useless
or effectively disabled. You'll also need to zero out the ALL_TRUSTED
rule for the same reason.


 I am making the assumption that our clients are mostly sending 'clean'
 email (I know, I am trusting *a lot* my users but nevertheless).
 
 So the question is: how efficient will be SA without using the bayesian
 classifier? Are all the remaining rulesets (apart from BAYES_*)
 sufficient to shave off spam email?

Define spam.

Running SA on your outgoing SMTP will not catch botnet generated junk,
neither spam nor malware. This would require sniffing raw traffic. Or
completely firewalling off outgoing port 25 connections.

You explicitly mention your users (corporate or home?) sending mail.
Are you talking about them possibly running bulk sending services, or
hand crafted unsolicited mail to individual recipients?

Unless there's a 419 gang operating from your internal network, there
might not be much left for SA with stock rules to classify spam...


That said, it is entirely possible to run SA without the Bayesian
classifier. There's an option to disable it, and different score sets
are used generated specifically for this case.


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}