RE: Bug in iXhash plugin - fixed version available

[RobertH]

is there anything wrong with still using an older pre 1.5.x version of
iXhash?

is there a problem that makes an upgrade recommended?

OR

is there a problem that forces up to upgrade?

 - rh

Re: Running message through a single SA test

[Matt Kettler]

Kelly Jones wrote:
> I want to run a message through ONE SpamAssassin test w/o the overhead
> of running all the tests.
>
> I realize many SA tests are just regexs (so I could use procmail or
> something), but this test is a meta test and it may change from time
> to time.
>
> Does SA have a "--run-just-this-test=FOO" option?
>
>   
No, but you can use -c to change the config directory to something other
than the one containing the default ruleset, and instead contains a
single .cf file with one rule.

Re: bohunu

[Chris]

On Wednesday 03 December 2008 7:01 pm, Michael Hutchinson wrote:

>
> Hello,
>
> I was using Pyzor until about 2 months ago. It was quite good then, I
> don't think I ever got a False Positive with it, and it did stop a lot
> of Spam - not as much as Razor, but still significant. I had to take it
> offline as I was getting timeouts doing E-Mail scanning. I have not
> tried the new version yet - I badly want to, but our Mail server sits on
> Debian Sarge, and there is no way I can run the Binary of Bohuno as it
> requires a version of SSL I cannot use in Sarge.
> Hopefully someone can try it on a more recent distro, and provide some
> information as to whether it is any good or not.
>
> Cheers,
> Mike

The old Pyzor is still working at least for me:

X-Spam-Pyzor: Reported 20 times.

Are you using this in your 'servers' file

82.94.255.100:24441
[EMAIL PROTECTED] ~]$ pyzor ping
82.94.255.100:24441 (200, 'OK')

-- 
Chris
KeyID 0xE372A7DA98E6705C


pgpbni151BJky.pgp
Description: PGP signature

Re: installing sanesecurity

[Daryl C. W. O'Shea]

On 03/12/2008 9:06 PM, Karsten Bräckelmann wrote:
>>> Darly posted a very similar rule to this a while ago, triggering on the
>>> strange cid- prefix in the live spaces URI. You can use that just as
>>> well.
>> Thanks I will give that rule a shot and check out the earlier post by Darly.
> 
> Whoops. :)  Daryl C. W. O'Shea I mean...  Sorry Daryl. Would that be ok
> as a pet-name? ;)

Sorry, a high school science teacher of mine (Phil Stoesser... Physics
with Phil) beat you to that one a long time ago.

Daryl

Re: installing sanesecurity

[Karsten Bräckelmann]

> > Darly posted a very similar rule to this a while ago, triggering on the
> > strange cid- prefix in the live spaces URI. You can use that just as
> > well.
> 
> Thanks I will give that rule a shot and check out the earlier post by Darly.

Whoops. :)  Daryl C. W. O'Shea I mean...  Sorry Daryl. Would that be ok
as a pet-name? ;)


> I appreciate your assitance and your patience.

-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: installing sanesecurity

[Karsten Bräckelmann]

On Thu, 2008-12-04 at 02:26 +0100, Karsten Bräckelmann wrote:
> On Thu, 2008-12-04 at 13:48 +1300, Kate wrote:

> > Yeah have been getting lots of variations of: 
> > http://www.pastebin.ca/1275436
> > Quite a lot are getting caught but in saying that alot are still getting 
> > through.
> 
> That one example smells like pure spam to me. Not phish, definitely not
> a scam (though I didn't investigate much).
> 
> Funnily enough, the Sanesecurity.Spam.9216 found in the *scam* sigs [1]
> does match. However, it translates to the RE
>   m~http://cid-.{0,30}\.spaces\.live\.com/blog/cns~
> 
> This topic has been beaten to death recently...

More on-topic. More beating dead horses. :)  We've discussed this very
spam type recently. Scores around 10+ here...


They usually hit at least RCVD_IN_XBL, if not a few more.

They hit any custom rule for the live spaces URI, including the one
above as per SaneSecurity scam sigs, Daryls, and a custom one I am
running locally, targeting the alphanumeric alternation.

They all are direct MUA to MX transmissions, no relay.

That spample (like most of these I have seen) hit RCVD_IN_BRBL (which
has been discussed a few times recently, too) and also hits the DNSBL
RCVD_IN_NIXSPAM, which can be found as an *additional* info on the
iXhash plugin pages [2]. It does not use that hash but sending IPs,
though.

Oh, yeah, also all of those I have seen do hit a rather cute rule of
mine, which can be found in my sandbox.

rawbody  __PQRTW_4_A m,\s*,
rawbody  __PQRTW_4_SPAN  m,\s*,
meta PQRTW_4 __PQRTW_4_A || __PQRTW_4_SPAN
scorePQRTW_4 1.0

That score is rather conservative, FWIW.  And I sure hope the spammers
stopped reading this thread like 10 posts ago... I love that rule. :-)

  guenther


[1] Which I coincidentally just this evening started to look into for an
entirely unrelated reason.
[2] http://ixhash.net/

-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: installing sanesecurity

[Lists]

mouss wrote:

Lists a écrit :
  

Karsten Bräckelmann wrote:


Thank you for the information I will attempt to get it up an running,
have had a huge increase in spam last week or so and just trying to
get it under control.



What type of *spam* are you referring to that you want to kill by
throwing anti-virus signatures at them? Are all of them phishing or
scam?

Hey, you said spam. We might be back on-topic, however gray! ;)

  
  

Yeah have been getting lots of variations of:
http://www.pastebin.ca/1275436
Quite a lot are getting caught but in saying that alot are still getting
through.




in your postfix, add
reject_rbl_client zen.spamhaus.org
to your smtpd_recipient_restrictions (after reject_unauth_destination).
  

Thanks, I have added this - does this look up zen.spamhaus.org and match 
against a list there?

Running message through a single SA test

[Kelly Jones]

I want to run a message through ONE SpamAssassin test w/o the overhead
of running all the tests.

I realize many SA tests are just regexs (so I could use procmail or
something), but this test is a meta test and it may change from time
to time.

Does SA have a "--run-just-this-test=FOO" option?

-- 
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.

Re: installing sanesecurity

[Lists]

Karsten Bräckelmann wrote:

On Thu, 2008-12-04 at 13:48 +1300, Lists wrote:
  

Karsten Bräckelmann wrote:



  

What type of *spam* are you referring to that you want to kill by
throwing anti-virus signatures at them? Are all of them phishing or
scam?

Hey, you said spam. We might be back on-topic, however gray! ;)
  
Yeah have been getting lots of variations of: 
http://www.pastebin.ca/1275436
Quite a lot are getting caught but in saying that alot are still getting 
through.



That one example smells like pure spam to me. Not phish, definitely not
a scam (though I didn't investigate much).

Funnily enough, the Sanesecurity.Spam.9216 found in the *scam* sigs [1]
does match. However, it translates to the RE
  m~http://cid-.{0,30}\.spaces\.live\.com/blog/cns~

This topic has been beaten to death recently...


  
Sorry for the 'idiot' questions its just that I am a very windows based 
person who is now looking after a linux system and I struggle at times 
to get my head around some of the concepts.



No problem, as long as we're staying on-topic. ;)  Anyway, something
most new-ish users tend to get wrong is asking the right questions. Why
didn't you just ask how to catch these providing in example in the first
place, rather than asking something strange you *guessed* might help...
  
Yeah I had done a bit of googling and reading on the list and it seemed 
the sanesecurity for clamav was a good option to try.

I think I will still look into using it at some stage.

I you want to get your ClamAV on steroids -- sure, go ahead. If you want
to catch that spam, a trivial SA rule will do.


Back to that spam. I assume they are all quite similar in design, text,
and the spaces.live.com URI?

You can *easily* get the result of that SaneSecurity scam sig in SA.

uri  SANESEC_9216  m~http://cid-.{0,30}\.spaces\.live
\.com/blog/cns~
scoreSANESEC_9216  5.0
describe SANESEC_9216  SaneSecurity.Spam.9216

There you go. Including a kill-level score for that rule, just like the
ClamAV third-party sig would have resulted in. Note though that I don't
advice to use that high a score. (Didn't --lint check the rule either,
mind you. ;)


Darly posted a very similar rule to this a while ago, triggering on the
strange cid- prefix in the live spaces URI. You can use that just as
well.
  

Thanks I will give that rule a shot and check out the earlier post by Darly.
I appreciate your assitance and your patience.
Kate


  

Nope didn't mean to send it to you before sorry.



I asked, because I would have forwarded (parts) to the list anyway. :)


  

Re: installing sanesecurity

[Karsten Bräckelmann]

On Thu, 2008-12-04 at 13:48 +1300, Lists wrote:
> Karsten Bräckelmann wrote:

> > What type of *spam* are you referring to that you want to kill by
> > throwing anti-virus signatures at them? Are all of them phishing or
> > scam?
> >
> > Hey, you said spam. We might be back on-topic, however gray! ;)
> 
> Yeah have been getting lots of variations of: 
> http://www.pastebin.ca/1275436
> Quite a lot are getting caught but in saying that alot are still getting 
> through.

That one example smells like pure spam to me. Not phish, definitely not
a scam (though I didn't investigate much).

Funnily enough, the Sanesecurity.Spam.9216 found in the *scam* sigs [1]
does match. However, it translates to the RE
  m~http://cid-.{0,30}\.spaces\.live\.com/blog/cns~

This topic has been beaten to death recently...


> Sorry for the 'idiot' questions its just that I am a very windows based 
> person who is now looking after a linux system and I struggle at times 
> to get my head around some of the concepts.

No problem, as long as we're staying on-topic. ;)  Anyway, something
most new-ish users tend to get wrong is asking the right questions. Why
didn't you just ask how to catch these providing in example in the first
place, rather than asking something strange you *guessed* might help...

I you want to get your ClamAV on steroids -- sure, go ahead. If you want
to catch that spam, a trivial SA rule will do.


Back to that spam. I assume they are all quite similar in design, text,
and the spaces.live.com URI?

You can *easily* get the result of that SaneSecurity scam sig in SA.

uri  SANESEC_9216  m~http://cid-.{0,30}\.spaces\.live
\.com/blog/cns~
scoreSANESEC_9216  5.0
describe SANESEC_9216  SaneSecurity.Spam.9216

There you go. Including a kill-level score for that rule, just like the
ClamAV third-party sig would have resulted in. Note though that I don't
advice to use that high a score. (Didn't --lint check the rule either,
mind you. ;)


Darly posted a very similar rule to this a while ago, triggering on the
strange cid- prefix in the live spaces URI. You can use that just as
well.


> Nope didn't mean to send it to you before sorry.

I asked, because I would have forwarded (parts) to the list anyway. :)


-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: skew the AWL on spam report

[Matt Kettler]

mouss wrote:
> Matt Kettler a écrit :
>   
>> mouss wrote:
>> 
>>> Matt Kettler a écrit :
>>>   
>>>   
 Brian J. Murrell wrote:
 
 
> If I get a spam and I need to have SA learn that it's spam with
> sa-learn, wouldn't it be useful to also skew the AWL for that sender so
> that future uses of the AWL for that spammer will push the overall spam
> score up?
>
> Thots?
>   
>   
>   
 If a spammer is using the same sending address over and over again,
 blacklist them entirely.

 That said, I've never seen a spammer re-use the same address twice.
 
 
>>> My understanding is "the other side". you get a spam and awl gives it a
>>> negative score. you run sa-learn and you want this to "nuke" the awl
>>> entry because if awl gives a too negative score, then sa-learn is
>>> useless (unless BAYES_99 is set to a very high value).
>>>
>>>   
>>>   
>> That sounds like you have a broken trust path. It seems unlikely you'd
>> have gotten nonspam from the same address *AND* IP address before.
>>
>> 
>
> I am thinking about this case: Joe the spammer bombs you with mail that
> is not detected as spam. he gets a negative awl.
That statement implies that there's a "score" for the user in the AWL.

The AWL score varies with what the current messages pre-awl score. The
AWL can think a sender has a +50 average, ie: strong spam, and if a
message comes in that scores +100, the AWL will set itself to -25.
However, if the same message was 0 before the AWL ran, it would give it +25.

Or were you talking about having a negative average because all the
messages sent as a bomb had negative scores?

>  so the questions are:
>
> - if user passes all the message to sa-learn, will that nuke the
> negative awl value?
>   
sa-learn doesn't touch the AWL. At all.
> - is it enough to pass few messages? (in short, does "manual" training
> have more "weight" than automatic awl learning?)
>   
There's no such thing as manual training of the AWL. Actually, there's
no such thing as "training" for it either.

The AWL averages scores. nothing more, nothing less. The message score
is added when the message is scanned. The AWL has no concept of spam or
not, just what the historical average is.

You can force fake messages with +100 scores in using spamassassin
--add-addr-to-blacklist, but that's not really "training" it's just
shoving the average around.

>
>   

Re: installing sanesecurity

[mouss]

Lists a écrit :
> Karsten Bräckelmann wrote:
>>> Thank you for the information I will attempt to get it up an running,
>>> have had a huge increase in spam last week or so and just trying to
>>> get it under control.
>>> 
>>
>> What type of *spam* are you referring to that you want to kill by
>> throwing anti-virus signatures at them? Are all of them phishing or
>> scam?
>>
>> Hey, you said spam. We might be back on-topic, however gray! ;)
>>
>>   
> Yeah have been getting lots of variations of:
> http://www.pastebin.ca/1275436
> Quite a lot are getting caught but in saying that alot are still getting
> through.
> 

in your postfix, add
reject_rbl_client zen.spamhaus.org
to your smtpd_recipient_restrictions (after reject_unauth_destination).

> Sorry for the 'idiot' questions its just that I am a very windows based
> person who is now looking after a linux system and I struggle at times
> to get my head around some of the concepts.
> 
> Thanks for you help
> Kate
> 
> Nope didn't mean to send it to you before sorry.

Re: skew the AWL on spam report

[mouss]

Matt Kettler a écrit :
> mouss wrote:
>> Matt Kettler a écrit :
>>   
>>> Brian J. Murrell wrote:
>>> 
 If I get a spam and I need to have SA learn that it's spam with
 sa-learn, wouldn't it be useful to also skew the AWL for that sender so
 that future uses of the AWL for that spammer will push the overall spam
 score up?

 Thots?
   
   
>>> If a spammer is using the same sending address over and over again,
>>> blacklist them entirely.
>>>
>>> That said, I've never seen a spammer re-use the same address twice.
>>> 
>> My understanding is "the other side". you get a spam and awl gives it a
>> negative score. you run sa-learn and you want this to "nuke" the awl
>> entry because if awl gives a too negative score, then sa-learn is
>> useless (unless BAYES_99 is set to a very high value).
>>
>>   
> That sounds like you have a broken trust path. It seems unlikely you'd
> have gotten nonspam from the same address *AND* IP address before.
> 

I am thinking about this case: Joe the spammer bombs you with mail that
is not detected as spam. he gets a negative awl. so the questions are:

- if user passes all the message to sa-learn, will that nuke the
negative awl value?

- is it enough to pass few messages? (in short, does "manual" training
have more "weight" than automatic awl learning?)

RE: bohunu

[Michael Hutchinson]

> -Original Message-
> From: Niels Przybilla [mailto:[EMAIL PROTECTED]
> Sent: 3 December 2008 6:01 p.m.
> To: users@spamassassin.apache.org
> Subject: bohunu
> 
> Hi,
> 
> is somebody here using bohunu.com
> 
> Is it worth testing it ?
> 
> BR Niels


Hello,

I was using Pyzor until about 2 months ago. It was quite good then, I
don't think I ever got a False Positive with it, and it did stop a lot
of Spam - not as much as Razor, but still significant. I had to take it
offline as I was getting timeouts doing E-Mail scanning. I have not
tried the new version yet - I badly want to, but our Mail server sits on
Debian Sarge, and there is no way I can run the Binary of Bohuno as it
requires a version of SSL I cannot use in Sarge. 
Hopefully someone can try it on a more recent distro, and provide some
information as to whether it is any good or not.

Cheers,
Mike

Re: dkim update:

[Mark Martinec]

Michael,

> I am completing some testing on new altermime version 0.3.10 for freebsd
> (it has already been submitted to ports)
>
> If you remember, using dkim signing and altermime would add \r\n to
> emails if you added disclaimers.
> (i have separate  plain text and html disclaimers)

Actually just a \r, and not always.

> Several emails to [EMAIL PROTECTED] and [EMAIL PROTECTED]
> with disclaimers seem to pass now, even with amavisd 2.6.1.

altermime 0.3.10 still stumbles with quote-printable encoding.
Using amavisd-new-2.6.2(-rc*) avoids the problem with lone \r
in altermime disclaimers.

> For anyone running dkim, this one should pass.

It does.

> on seperate note, if SpamAssassin is scoring based on dkim/ pass/fail,
> what extra functionality, other then amavisd dkim whitelisting do I get
> by using $enable_dkim_verification = 1 ?

- invokes DKIM verification regardless of mail size (SA is only invoked
  for smaller messages - below the configurable limit);

- can load a policy bank based on a verified signature through
  @author_to_policy_bank_maps - and a policy bank can affect most
  settings of your choice, e.g. whitelist spam or banned contents,
  add score points, affect quarantine, notifications, mail routing ...;

- adjust score through @signer_reputation_maps (using a formula
  similar to AWL);

- add a header field Authentication-Results:, helpful for troubleshooting
  and potentially useful for (future) MUAs; there is a draft standard on this;

- log information about DKIM/DK verification results to facilitate
  troubleshooting and gathering statistics.


If you don't need any of these, by all means, turn off the 
$enable_dkim_verification to save few milliseconds on
nonsigned mail and few tens of milliseconds on signed mail.
Also the verification code is not loaded (unless you do DKIM
signing), saving a little memory.

Regarding a DNS lookup on signed mail, even if you do
DKIM/DK verification in both amavisd and in SpamAssassin,
it is very likely the DNS result would be cached in your DNS,
so you pay the price of a DNS query delay only once.


  Mark

Re: skew the AWL on spam report

[Matt Kettler]

mouss wrote:
> Matt Kettler a écrit :
>   
>> Brian J. Murrell wrote:
>> 
>>> If I get a spam and I need to have SA learn that it's spam with
>>> sa-learn, wouldn't it be useful to also skew the AWL for that sender so
>>> that future uses of the AWL for that spammer will push the overall spam
>>> score up?
>>>
>>> Thots?
>>>   
>>>   
>> If a spammer is using the same sending address over and over again,
>> blacklist them entirely.
>>
>> That said, I've never seen a spammer re-use the same address twice.
>> 
>
> My understanding is "the other side". you get a spam and awl gives it a
> negative score. you run sa-learn and you want this to "nuke" the awl
> entry because if awl gives a too negative score, then sa-learn is
> useless (unless BAYES_99 is set to a very high value).
>
>   
That sounds like you have a broken trust path. It seems unlikely you'd
have gotten nonspam from the same address *AND* IP address before.

Re: installing sanesecurity

[Lists]

Karsten Bräckelmann wrote:
Thank you for the information I will attempt to get it up an running, 
have had a huge increase in spam last week or so and just trying to 
get it under control.



What type of *spam* are you referring to that you want to kill by
throwing anti-virus signatures at them? Are all of them phishing or
scam?

Hey, you said spam. We might be back on-topic, however gray! ;)

  
Yeah have been getting lots of variations of: 
http://www.pastebin.ca/1275436
Quite a lot are getting caught but in saying that alot are still getting 
through.


Sorry for the 'idiot' questions its just that I am a very windows based 
person who is now looking after a linux system and I struggle at times 
to get my head around some of the concepts.


Thanks for you help
Kate

Nope didn't mean to send it to you before sorry.

Re: installing sanesecurity

[Karsten Bräckelmann]

> Thank you for the information I will attempt to get it up an running, 
> have had a huge increase in spam last week or so and just trying to get 
> it under control.

What type of *spam* are you referring to that you want to kill by
throwing anti-virus signatures at them? Are all of them phishing or
scam?

Hey, you said spam. We might be back on-topic, however gray! ;)

-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: installing sanesecurity

[Karsten Bräckelmann]

On Thu, 2008-12-04 at 12:43 +1300, Lists wrote:
> Arthur Dent wrote:

> > The best thing to do is to download the script, put it somewhere where
> > the user that will run it (possibly "clamav") has read + execute access,
> > (I created a /home/clamav/ directory) and then try running it manually
> > first. If it works it will download the extra files need by Clamav for
> > the spam and phishing sigs.
> 
> So if the manual run works it will download everything needed and clam 
> will know its there and to use it?

Depends.  Come on, Kate, have a look at the scripts. As I briefly
mentioned before, they are intended to be read (at least those I ever
had a look at include their own full docs) and *configured*.

The latter is important. If it isn't configured according to *your*
ClamAV setup, it can't possibly do anything. If and how the script or
your cron job will have to poke clamd (you're running that, aren't you?)
again, depends.

> Also will it add info to the headers of the email so that I can  check 
> which emails are being hit by the plugin?

No, "it" doesn't.

It's third-party signatures. ClamAV uses them, if configured properly.
Whatever adds headers *now* will do so for third-party sigs just as
well.

Wait -- what "plugin" are you talking about?


/me mumbles something about "wrong list" and "should have included TM
hints" according to some recent post on that other list...

-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Message size limit for sa-learn (oops)

[Wolfgang Zeikat]

Wolfgang Zeikat wrote:
We have set -s for spamc to 350k - and we can use spamassassin -t on 
messages of that size, but we can not sa-learn them, sa-learn -D -t puts 
out:


Sorry, it's late here. What I meant is

sa-learn -D --spam puts out:



[17460] info: archive-iterator: skipping large message
Learned tokens from 0 message(s) (0 message(s) examined)

Can we pass the 350k limit to sa-learn somehow?

Regards,

wolfgang

Re: installing sanesecurity

[Lists]

Arthur Dent wrote:

On Thu, Dec 04, 2008 at 09:49:23AM +1300, Lists wrote:
  

Hi all,

I am wanting to implement the sanesecurity addins to clamav but i am a  
bit lost.

I am running CentOS5 MailScanner Spamassassin ClamAV

Do I download the download scripts from  
http://www.sanesecurity.com/clamav/usage.htm

or do I go to the downloads page? (they seem to be different)

Once I have downloaded them I rename them to .sh then I run it and it  
installs itself including the cron job?


Is this correct - I feel I may be missing a few things.

Thanks
Kate



The best thing to do is to download the script, put it somewhere where
the user that will run it (possibly "clamav") has read + execute access,
(I created a /home/clamav/ directory) and then try running it manually
first. If it works it will download the extra files need by Clamav for
the spam and phishing sigs.
  
So if the manual run works it will download everything needed and clam 
will know its there and to use it?
Also will it add info to the headers of the email so that I can  check 
which emails are being hit by the plugin?

If the manual run works without errors, then add it to your (root's)
crontab (unless you want to put in cron.daily).

# crontab -e
[puts you into your default editor - in my case vim - if you have vim
too do the following - otherwise you'll have to find out how your editor
works]
i 
[to go into insert mode]

17 2 * * * /home/clamav/update_sanesecurity.sh
[this creates a cron job that will run at 17 minutes past 2 every night]
press Escape and then 
:wq

[ie colon then w(rite) and q(uit) - this saves the crontab file]

You should now be all set to receive daily SaneSecurity updates...

HTH

AD

  

Thanks am looking foward to getting this up and running.

Kate

Re: installing sanesecurity

[Lists]

Karsten Bräckelmann wrote:
I am wanting to implement the sanesecurity addins to clamav but i am a 
bit lost.

I am running CentOS5 MailScanner Spamassassin ClamAV



Kate, this is the wrong mailing list. The ClamAV users list comes
closest for third-party ClamAV (sic) signatures without a list of their
own.
  

Ok, sorry about that
  
Do I download the download scripts from 
http://www.sanesecurity.com/clamav/usage.htm

or do I go to the downloads page? (they seem to be different)



The downloads page offers the latest sig files themselfs -- just in case
one needs a snapshot. That page is *not* suitable for periodically
updates.

You need to follow the *usage* instructions and get a script that
performs the actual download, usually run by cron. Details (how to call
the update script and how to configure it to your needs) can be found
alongside the respective scripts.

  
Once I have downloaded them I rename them to .sh then I run it and it 
installs itself including the cron job?



No, the scripts will download the latest signatures -- you need to take
care about the cron job.

The reason these need to be re-named to .sh most likely is, so you can
conveniently read them in your browser as text/plain. Try it, click one
of the scripts' links you like...


  
Thank you for the information I will attempt to get it up an running, 
have had a huge increase in spam last week or so and just trying to get 
it under control.

Cheers for the help
Kate

Message size limit for sa-learn

[Wolfgang Zeikat]

We have set -s for spamc to 350k - and we can use spamassassin -t on 
messages of that size, but we can not sa-learn them, sa-learn -D -t puts 
out:


[17460] info: archive-iterator: skipping large message
Learned tokens from 0 message(s) (0 message(s) examined)

Can we pass the 350k limit to sa-learn somehow?

Regards,

wolfgang

Re: skew the AWL on spam report

[mouss]

Matt Kettler a écrit :
> Brian J. Murrell wrote:
>> If I get a spam and I need to have SA learn that it's spam with
>> sa-learn, wouldn't it be useful to also skew the AWL for that sender so
>> that future uses of the AWL for that spammer will push the overall spam
>> score up?
>>
>> Thots?
>>   
> 
> If a spammer is using the same sending address over and over again,
> blacklist them entirely.
> 
> That said, I've never seen a spammer re-use the same address twice.

My understanding is "the other side". you get a spam and awl gives it a
negative score. you run sa-learn and you want this to "nuke" the awl
entry because if awl gives a too negative score, then sa-learn is
useless (unless BAYES_99 is set to a very high value).

Re: Bug in iXhash plugin - fixed version available

[Aaron Wolfe]

On Wed, Dec 3, 2008 at 1:57 PM, Arthur Dent <[EMAIL PROTECTED]> wrote:
> On Wed, Dec 03, 2008 at 01:08:32PM -0500, Rose, Bobby wrote:
>> I just tried again with this 1.5.2 version and on box it times out querying 
>> and on another it seems to run but no hits again.  Both my boxes are SA3.2.5.
>>
>> Does anyone have a message that is known to have hashes on any of iXhash 
>> hosts?
>
> Well actually the one I posted earlier in this thread used to hit
> iXhash; failed on v. 1.5 and v. 1.5.1 but, I'm pleased to say hits (for
> me at least) with the latest version (v. 1.5.2).
>
> You can try it here:
>
> http://pastebin.ca/1269211
>
> And I would like to take this opportunity to thank Dirk and those who
> helped him (Karsten et al) to track down the bug for all their efforts
> to get this great little plugin working to the max once more.
>

As of 1.5.2 we are getting hits again here.  Thanks to everyone who
helped with this!


> Thanks!
>
> AD
>
>>
>

Re: installing sanesecurity

[Karsten Bräckelmann]

> I am wanting to implement the sanesecurity addins to clamav but i am a 
> bit lost.
> I am running CentOS5 MailScanner Spamassassin ClamAV

Kate, this is the wrong mailing list. The ClamAV users list comes
closest for third-party ClamAV (sic) signatures without a list of their
own.

> Do I download the download scripts from 
> http://www.sanesecurity.com/clamav/usage.htm
> or do I go to the downloads page? (they seem to be different)

The downloads page offers the latest sig files themselfs -- just in case
one needs a snapshot. That page is *not* suitable for periodically
updates.

You need to follow the *usage* instructions and get a script that
performs the actual download, usually run by cron. Details (how to call
the update script and how to configure it to your needs) can be found
alongside the respective scripts.

> Once I have downloaded them I rename them to .sh then I run it and it 
> installs itself including the cron job?

No, the scripts will download the latest signatures -- you need to take
care about the cron job.

The reason these need to be re-named to .sh most likely is, so you can
conveniently read them in your browser as text/plain. Try it, click one
of the scripts' links you like...


-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: installing sanesecurity

[Arthur Dent]

On Thu, Dec 04, 2008 at 09:49:23AM +1300, Lists wrote:
> Hi all,
>
> I am wanting to implement the sanesecurity addins to clamav but i am a  
> bit lost.
> I am running CentOS5 MailScanner Spamassassin ClamAV
>
> Do I download the download scripts from  
> http://www.sanesecurity.com/clamav/usage.htm
> or do I go to the downloads page? (they seem to be different)
>
> Once I have downloaded them I rename them to .sh then I run it and it  
> installs itself including the cron job?
>
> Is this correct - I feel I may be missing a few things.
>
> Thanks
> Kate

The best thing to do is to download the script, put it somewhere where
the user that will run it (possibly "clamav") has read + execute access,
(I created a /home/clamav/ directory) and then try running it manually
first. If it works it will download the extra files need by Clamav for
the spam and phishing sigs.

If the manual run works without errors, then add it to your (root's)
crontab (unless you want to put in cron.daily).

# crontab -e
[puts you into your default editor - in my case vim - if you have vim
too do the following - otherwise you'll have to find out how your editor
works]
i 
[to go into insert mode]
17 2 * * * /home/clamav/update_sanesecurity.sh
[this creates a cron job that will run at 17 minutes past 2 every night]
press Escape and then 
:wq
[ie colon then w(rite) and q(uit) - this saves the crontab file]

You should now be all set to receive daily SaneSecurity updates...

HTH

AD



pgpi0uiQtSwvm.pgp
Description: PGP signature

installing sanesecurity

[Lists]

Hi all,

I am wanting to implement the sanesecurity addins to clamav but i am a 
bit lost.

I am running CentOS5 MailScanner Spamassassin ClamAV

Do I download the download scripts from 
http://www.sanesecurity.com/clamav/usage.htm

or do I go to the downloads page? (they seem to be different)

Once I have downloaded them I rename them to .sh then I run it and it 
installs itself including the cron job?


Is this correct - I feel I may be missing a few things.

Thanks
Kate

I hate one certain language

[jidanni]

Never mind the below, I solved it with
header J_CHSET3 
Subject:raw=~/\s=\?(windows-(125[0125]|874)|koi8-r|GB2312|iso-8859-[28])\?/i

The below:
Here we go again.
How can I filter on
X-Spam-Languages: zh.gb2312
run it through spamassassin a second time?
Use _LANGUAGES_ somehow in a regexp?
Of course the LANGUAGE OPTIONS part of the man page just begs the
question of how to mark one as bad, instead of good. But never mind.
That is a never ending argument that I have forgotten.
Note I love other zh.*, just not zh.gb2312.
Hmm, I see I already do
ifplugin Mail::SpamAssassin::Plugin::TextCat
 # ok_languages en zh.big5
 # http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5697
 ok_languages en zh
 add_header all Languages _LANGUAGES_
 score UNWANTED_LANGUAGE_BODY 5
endif
ok_locales en zh

OK, solved as at top. Thanks. Bye.

Re: Bug in iXhash plugin - fixed version available

[Arthur Dent]

On Wed, Dec 03, 2008 at 01:08:32PM -0500, Rose, Bobby wrote:
> I just tried again with this 1.5.2 version and on box it times out querying 
> and on another it seems to run but no hits again.  Both my boxes are SA3.2.5.
> 
> Does anyone have a message that is known to have hashes on any of iXhash 
> hosts?

Well actually the one I posted earlier in this thread used to hit
iXhash; failed on v. 1.5 and v. 1.5.1 but, I'm pleased to say hits (for
me at least) with the latest version (v. 1.5.2).

You can try it here:

http://pastebin.ca/1269211

And I would like to take this opportunity to thank Dirk and those who
helped him (Karsten et al) to track down the bug for all their efforts
to get this great little plugin working to the max once more.

Thanks!

AD

> 


pgpRCByYNpMyJ.pgp
Description: PGP signature

dkim update:

[Michael Scheidell]

I am completing some testing on new altermime version 0.3.10 for freebsd 
(it has already been submitted to ports)


If you remember, using dkim signing and altermime would add \r\n to 
emails if you added disclaimers.


(i have separate  plain text and html disclaimers)

Several emails to [EMAIL PROTECTED] and [EMAIL PROTECTED] 
with disclaimers seem to pass now, even with amavisd 2.6.1.


For anyone running dkim, this one should pass.

on seperate note, if SpamAssassin is scoring based on dkim/ pass/fail, 
what extra functionality, other then amavisd dkim whitelisting do I get 
by using $enable_dkim_verification = 1 ?


--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
> *| *SECNAP Network Security Corporation

   * Certified SNORT Integrator
   * King of Spam Filters, SC Magazine 2008
   * Information Security Award 2008, Info Security Products Guide
   * CRN Magazine Top 40 Emerging Security Vendors

_
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.secnap.com/products/spammertrap/

_

SAGrey plugin (was: Re: skew the AWL on spam report)

[Karsten Bräckelmann]

On Wed, 2008-12-03 at 17:38 +, Nigel Frankcom wrote:
> Is Mail::SpamAssassin::Plugin::SAGrey part of the stat SA set? Neither
> yum nor CPAN seem to be able to find it here... though that could
> easily be down to user error.

Google finds it quite easily. ;)

  http://wiki.apache.org/spamassassin/CustomPlugins
  http://www.ehall.family-and-friends.us/software/spamassassin/sagrey/

> Hasn't appeared in sa-update either from what I've seen.

Not part of the official SA code, it's a custom plugin. Oh, and plugins
are unlikely to be distributed / updated using sa-update anyway, unless
someone offers a third-party channel dedicated to such things.

-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

RE: Bug in iXhash plugin - fixed version available

[Rose, Bobby]

I just tried again with this 1.5.2 version and on box it times out querying and 
on another it seems to run but no hits again.  Both my boxes are SA3.2.5.

Does anyone have a message that is known to have hashes on any of iXhash hosts?

-Original Message-
From: Giampaolo Tomassoni [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, December 03, 2008 12:49 PM
To: 'Marc Perkel'; 'Dirk Bonengel'
Cc: users@spamassassin.apache.org
Subject: RE: Bug in iXhash plugin - fixed version available

> -Original Message-
> From: Marc Perkel [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, December 03, 2008 12:04 AM
> 
> it's WORKING

Well,

it hangs my SA 3.2.4 setup on waiting for a reply from ctyme.ixhash.net .

The strange thing is that it consumes a lot of CPU while hanging... Some
problem in the ctyme.ixhash.net side? Anybody is experiencing the same?

Giampaolo


> Dirk Bonengel wrote:
> > OK, I found the bug.
> >
> > I just released a fixed release. Thanks to Lars Uhlmann for finding
> > the culprit and delivering a fix.
> > Problem was the regular expression checking the IP returned if it
> > belongs to the 127.x.x.x range.
> >
> > Hmm, I had this working before
> >
> > Soryy again for the trouble
> >
> > Dirk
> >
> >
> >
> >
> >
> >
> >

RE: Bug in iXhash plugin - fixed version available

[Giampaolo Tomassoni]

> -Original Message-
> From: Marc Perkel [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, December 03, 2008 12:04 AM
> 
> it's WORKING

Well,

it hangs my SA 3.2.4 setup on waiting for a reply from ctyme.ixhash.net .

The strange thing is that it consumes a lot of CPU while hanging... Some
problem in the ctyme.ixhash.net side? Anybody is experiencing the same?

Giampaolo


> Dirk Bonengel wrote:
> > OK, I found the bug.
> >
> > I just released a fixed release. Thanks to Lars Uhlmann for finding
> > the culprit and delivering a fix.
> > Problem was the regular expression checking the IP returned if it
> > belongs to the 127.x.x.x range.
> >
> > Hmm, I had this working before
> >
> > Soryy again for the trouble
> >
> > Dirk
> >
> >
> >
> >
> >
> >
> >

Re: skew the AWL on spam report

[Nigel Frankcom]

On Wed, 3 Dec 2008 09:56:58 -0500, Jeff Mincy <[EMAIL PROTECTED]>
wrote:

>   From: Matt Kettler <[EMAIL PROTECTED]>
>   Date: Tue, 02 Dec 2008 23:48:57 -0500
>   
>   Brian J. Murrell wrote:
>   > If I get a spam and I need to have SA learn that it's spam with
>   > sa-learn, wouldn't it be useful to also skew the AWL for that sender so
>   > that future uses of the AWL for that spammer will push the overall spam
>   > score up?
>   > Thots?
>
>You can use spamassassin --add-to-blacklist.   There isn't much of a
>point though, since the email address isn't likely to ever be reused.
>Only 5% of my spam is in the AWL.
>   
>   If a spammer is using the same sending address over and over again,
>   blacklist them entirely.
>   
>Yep.
>
>   That said, I've never seen a spammer re-use the same address twice.
>
>The sagrey plugin addresses this.   Sagrey hits on the 95% of
>spam that is from a new email+IP.
>
>-jeff


Is Mail::SpamAssassin::Plugin::SAGrey part of the stat SA set? Neither
yum nor CPAN seem to be able to find it here... though that could
easily be down to user error. Hasn't appeared in sa-update either from
what I've seen.

Nigel

Re: Detecting Porn photos

[Luis Daniel Lucio Quiroz]

Yes, Thanks


On Tuesday 02 December 2008 17:38:17 Kenneth Porter wrote:
> --On Thursday, November 27, 2008 10:44 PM -0600 Luis Daniel Lucio Quiroz
>
> <[EMAIL PROTECTED]> wrote:
> > I wonder if there is any module for SA to detect pornographic photos, not
> > only  OCR.
>
> How about setting up a system like the captcha-breakers, but in reverse?
> Instead of giving access to porn by breaking captchas, you let people vote
> on how good porn is and use the result in your spam filter. You need a
> website where your mail server sends the porn for evaluation and reads back
> the vote. You should get lots of volunteer visitors to "run the engine".

Re: Twist on Day Old Bread list idea

[Joseph Brennan]

--On Tuesday, December 2, 2008 12:23 -0800 Marc Perkel <[EMAIL PROTECTED]> 
wrote:




You query hostkarma.junkemailfilter.com

Not listed = new (new to us anyhow)
127.0.2.1 = last day
127.0.2.2 = last week
127.0.2.3 = older than a week

OK - so here's the rub. This catches 100% of all new domains. But - it
will have false positives because if an old domain has never emailed
anyone we filter for then it would also be considered new. We keep 40
days of data. So - this list might be useful as long as it was combined
with additional tests (probably spambot tests) as a score enhancer.



It's analogous to greylisting, to say that if we have not seen this
domain in the past N days, we tempfail, or score, or something.

However I think it would be better to have a software package that
implements this, rather than a remotely managed list, since each system
would have its own set of domains that it sees frequently (or that it
wants to whitelist permanently).

Joseph Brennan
Columbia University Information Technology

Re: skew the AWL on spam report

[Jeff Mincy]

   From: Matt Kettler <[EMAIL PROTECTED]>
   Date: Tue, 02 Dec 2008 23:48:57 -0500
   
   Brian J. Murrell wrote:
   > If I get a spam and I need to have SA learn that it's spam with
   > sa-learn, wouldn't it be useful to also skew the AWL for that sender so
   > that future uses of the AWL for that spammer will push the overall spam
   > score up?
   > Thots?

You can use spamassassin --add-to-blacklist.   There isn't much of a
point though, since the email address isn't likely to ever be reused.
Only 5% of my spam is in the AWL.
   
   If a spammer is using the same sending address over and over again,
   blacklist them entirely.
   
Yep.

   That said, I've never seen a spammer re-use the same address twice.

The sagrey plugin addresses this.   Sagrey hits on the 95% of
spam that is from a new email+IP.

-jeff

Re: Bad check_for_from_to_same code in EvalTests.pm?

[Theo Van Dinter]

On Wed, Dec 03, 2008 at 07:13:26AM -0700, Kelly Jones wrote:
> SA doesn't use EvalTests.pm's check_for_from_to_same test, but part of
> the code looks like this:

Wow.  Had to whip out the 3.1 code to find this...

> Is that right? Shouldn't the 'eq' be 'ne'?

As the comment about 6 lines up from there says:

# From and To have same address, but are not exactly the same and
# neither contains intermediate spaces.

:)

-- 
Randomly Selected Tagline:
"Before his State of the Union speech, the president's niece was arrested
 for trying to fill a fake prescription for the anti-anxiety drug Xanax. If
 you're not familiar with Xanax, the best way to describe it is, after
 taking three or four with a wine cooler, you become a really, really
 compassionate conservative."- Bill Maher, Politically Incorrect


pgpWbJxluAOBI.pgp
Description: PGP signature

Bad check_for_from_to_same code in EvalTests.pm?

[Kelly Jones]

SA doesn't use EvalTests.pm's check_for_from_to_same test, but part of
the code looks like this:

  return 0 if (!length($hdr_from) || !length($hdr_to) ||
   $hdr_from eq $hdr_to);

Is that right? Shouldn't the 'eq' be 'ne'?

-- 
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.

Re: skew the AWL on spam report

[Benny Pedersen]

On Wed, December 3, 2008 05:48, Matt Kettler wrote:

> That said, I've never seen a spammer re-use the same address twice.

i have :-)

olso why spf / dkim whitelist is the way to go, let spammers try to
get whitelisted

microsoft got it wroung with "Block Sender" :)


-- 
Benny Pedersen
Need more webspace ? http://www.servage.net/?coupon=cust37098

Re: Newbie Questions: Different Results for the same message

[Karsten Bräckelmann]

On Wed, 2008-12-03 at 03:36 -0800, Björn K wrote:
> Thank you, that should help. I don't really wanna print the whole headers
> here (not giving away too many internals on how which company's mails I
> handle in which way and what problems I have with it).

Forwarding mail for companies (smells like business to me) through GMX?
This seems to be your problem. But see below.

> It's a spamassassin 3.1.7 out of the Debian (Etch) repository (debian
> revision 2).
> 
> 
> Karsten Bräckelmann-2 wrote:

[ Cursing enumeration, there is only *one* such human, snipping an
utterly needless full quote including the sig. *sigh*  Nabble still
doesn't get it right. ]


> > For some better evaluation, we'd need the full X-Spam headers, both as
   ^^
> > inserted by your local SA on the first run *and* the manual second run.
> > Don't have that, so here's a guess.

I was asking for the SpamAssassin headers -- not all headers. *shrug*


-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: Newbie Questions: Different Results for the same message

[Björn K]

Thank you, that should help. I don't really wanna print the whole headers
here (not giving away too many internals on how which company's mails I
handle in which way and what problems I have with it).

It's a spamassassin 3.1.7 out of the Debian (Etch) repository (debian
revision 2).


Karsten Bräckelmann-2 wrote:
> 
> On Wed, 2008-12-03 at 02:00 -0800, Björn K wrote:
>> Hello,
>> 
>> I am relatively new to SpamAssassin and have some problems with email
>> which
>> seems to get completely different scores when I check them manually than
>> when the automatic check upon reception by the Exim mail server is
>> performed.
>> 
>> Before we use an own spam filter the mail was put into an imap folder for
>> an
>> external mail service to be read (GMX), filtered and forwarded back to
>> another mail box. That system is still working for parts. When a mail is
> 
> Despite mentioning IMAP folders -- I assume this involves forwarding to
> another SMTP or polling by GMX? If so, SA likely can not detect all this
> properly and thus tests some of these "internal" forwarding relays
> against blacklists, instead of the actually handing over external one.
> As a result, quite a lot of DNSBLs will not trigger and your SA performs
> less effective than it could.
> 
> You can fix this by tweaking trusted_networks and internal_networks. But
> that wasn't your question. :)
> 
>> transferred like this I can see the spam score being evaluated twice. For
>> example there was a mail containing only a link to dagwizhua -dot- com,
>> which is a bad address. It received 6.8 on first run, 3.6 on the second
>> run
>> only for a few additional headers added by the external mail service.
> 
> This difference might actually be due to the trust path outlined above.
> If GMX does polling, they could have correctly tested the external
> handing over relay against blacklists.
> 
> Your local run doesn't show any such hits.
> 
>> However, when I copied the mail into a text file and used spamc to send
>> it
>> the /same/ spamd process I got this result:
>> [EMAIL PROTECTED]:~$ LANG=C spamc -lR < spam-mail.txt | recode latin1..utf8
>> 12.9/5.0
> 
> For some better evaluation, we'd need the full X-Spam headers, both as
> inserted by your local SA on the first run *and* the manual second run.
> Don't have that, so here's a guess.
> 
>> Pkte Regelname  Beschreibung
>>  --
>> --
>>  0.6 NO_REAL_NAME   Kein vollständiger Name in Absendeadresse
>>  1.8 INVALID_DATE   Datumskopfzeile nicht standardkonform zu RFC
>> 2822
>>  0.0 UNPARSEABLE_RELAY  Informational: message has unparseable relay
>> lines
>>  1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Transportiert via Rechner in Liste von
>> www.spamcop.net
>>[Blocked - see
>> ]
> 
> This is about 3.6 (assuming some rounding), the score your first run
> ended up with.
> 
>>  3.3 URIBL_AB_SURBL Enthält URL in AB-Liste (www.surbl.org)
>> [URIs: dagwizhua -dot- com]
>>  2.6 URIBL_OB_SURBL Enthält URL in OB-Liste (www.surbl.org)
>> [URIs: dagwizhua -dot- com]
>>  3.6 URIBL_SC_SURBL Enthält URL in SC-Liste  (www.surbl.org)
>> [URIs: dagwizhua -dot- com]
> 
> These are moving targets. It is entirely possible that the URI
> blacklists haven't caught up when you initially scanned the mail -- and
> thus they didn't hit on the first run, but later only.
> 
>> -0.2 AWLAWL: From: address is in the auto white-list
> 
> Computed based on the sender/IP-block history.
> 
>> How can the results be so very different on the same spam process? Why
>> would
>> a few additional headers make a difference if the Bayes does not seem to
>> add
>> anything to the mail and there is no particular rule for those headers?
>> And
>> why does a manual scan produce a completely different result if the
>> service
>> that creates the actual results is the same process?
> 
> See above. It's likely not about the headers, but timing -- that URI
> simply hasn't been on the blacklists before.
> 
> The difference to the GMX score probably is due to the trust path. Plus
> the SA version used and thus the scores per rule. Don't remember
> off-hand which SA version GMX uses, but I do see you're running an old
> version, aren't you? The scores (and rules, mind you) don't match a
> recent SA 3.2.x.
> 
>   guenther
> 
> 
> -- 
> char
> *t="[EMAIL PROTECTED]";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i c<<=1:
> (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0;
> }}}
> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Newbie-Questions%3A-Different-Results-for-the-same-message-tp20809927p20811311.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: Newbie Questions: Different Results for the same message

[Kai Schaetzl]

Björn K wrote on Wed, 3 Dec 2008 02:00:32 -0800 (PST):

> How can the results be so very different on the same spam process?

Too many whys ;-) Comparing overall scores doesn't provide any insight.
You want to compare the rules that hit, then you'll see what is different 
(and most of the differences should then be self-explicatory).

In general the time a message gets scanned does make a difference if it 
comes to any network/distributed tests - e.g. the message parameters may 
not be known as spam by the various RBLs at the time of the first scan. 
And if you use different systems (what you seem to do, one internal, one 
external), then there's a great chance, that they are configured 
different, of course.

Kai

-- 
Kai Schätzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

Re: Newbie Questions: Different Results for the same message

[Karsten Bräckelmann]

On Wed, 2008-12-03 at 02:00 -0800, Björn K wrote:
> Hello,
> 
> I am relatively new to SpamAssassin and have some problems with email which
> seems to get completely different scores when I check them manually than
> when the automatic check upon reception by the Exim mail server is
> performed.
> 
> Before we use an own spam filter the mail was put into an imap folder for an
> external mail service to be read (GMX), filtered and forwarded back to
> another mail box. That system is still working for parts. When a mail is

Despite mentioning IMAP folders -- I assume this involves forwarding to
another SMTP or polling by GMX? If so, SA likely can not detect all this
properly and thus tests some of these "internal" forwarding relays
against blacklists, instead of the actually handing over external one.
As a result, quite a lot of DNSBLs will not trigger and your SA performs
less effective than it could.

You can fix this by tweaking trusted_networks and internal_networks. But
that wasn't your question. :)

> transferred like this I can see the spam score being evaluated twice. For
> example there was a mail containing only a link to dagwizhua -dot- com,
> which is a bad address. It received 6.8 on first run, 3.6 on the second run
> only for a few additional headers added by the external mail service.

This difference might actually be due to the trust path outlined above.
If GMX does polling, they could have correctly tested the external
handing over relay against blacklists.

Your local run doesn't show any such hits.

> However, when I copied the mail into a text file and used spamc to send it
> the /same/ spamd process I got this result:
> [EMAIL PROTECTED]:~$ LANG=C spamc -lR < spam-mail.txt | recode latin1..utf8
> 12.9/5.0

For some better evaluation, we'd need the full X-Spam headers, both as
inserted by your local SA on the first run *and* the manual second run.
Don't have that, so here's a guess.

> Pkte Regelname  Beschreibung
>  -- --
>  0.6 NO_REAL_NAME   Kein vollständiger Name in Absendeadresse
>  1.8 INVALID_DATE   Datumskopfzeile nicht standardkonform zu RFC 2822
>  0.0 UNPARSEABLE_RELAY  Informational: message has unparseable relay lines
>  1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Transportiert via Rechner in Liste von
> www.spamcop.net
>[Blocked - see ]

This is about 3.6 (assuming some rounding), the score your first run
ended up with.

>  3.3 URIBL_AB_SURBL Enthält URL in AB-Liste (www.surbl.org)
> [URIs: dagwizhua -dot- com]
>  2.6 URIBL_OB_SURBL Enthält URL in OB-Liste (www.surbl.org)
> [URIs: dagwizhua -dot- com]
>  3.6 URIBL_SC_SURBL Enthält URL in SC-Liste  (www.surbl.org)
> [URIs: dagwizhua -dot- com]

These are moving targets. It is entirely possible that the URI
blacklists haven't caught up when you initially scanned the mail -- and
thus they didn't hit on the first run, but later only.

> -0.2 AWLAWL: From: address is in the auto white-list

Computed based on the sender/IP-block history.

> How can the results be so very different on the same spam process? Why would
> a few additional headers make a difference if the Bayes does not seem to add
> anything to the mail and there is no particular rule for those headers? And
> why does a manual scan produce a completely different result if the service
> that creates the actual results is the same process?

See above. It's likely not about the headers, but timing -- that URI
simply hasn't been on the blacklists before.

The difference to the GMX score probably is due to the trust path. Plus
the SA version used and thus the scores per rule. Don't remember
off-hand which SA version GMX uses, but I do see you're running an old
version, aren't you? The scores (and rules, mind you) don't match a
recent SA 3.2.x.

  guenther


-- 
char *t="[EMAIL PROTECTED]";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Newbie Questions: Different Results for the same message

[Björn K]

Hello,

I am relatively new to SpamAssassin and have some problems with email which
seems to get completely different scores when I check them manually than
when the automatic check upon reception by the Exim mail server is
performed.

Before we use an own spam filter the mail was put into an imap folder for an
external mail service to be read (GMX), filtered and forwarded back to
another mail box. That system is still working for parts. When a mail is
transferred like this I can see the spam score being evaluated twice. For
example there was a mail containing only a link to dagwizhua -dot- com,
which is a bad address. It received 6.8 on first run, 3.6 on the second run
only for a few additional headers added by the external mail service.

However, when I copied the mail into a text file and used spamc to send it
the /same/ spamd process I got this result:
[EMAIL PROTECTED]:~$ LANG=C spamc -lR < spam-mail.txt | recode latin1..utf8
12.9/5.0
Software zur Erkennung von "Spam" auf dem Rechner

 (...)

Inhaltsanalyse im Detail:   (12.9 Punkte, 5.0 benötigt)

Pkte Regelname  Beschreibung
 --
--
 0.6 NO_REAL_NAME   Kein vollständiger Name in Absendeadresse
 1.8 INVALID_DATE   Datumskopfzeile nicht standardkonform zu RFC
2822
 0.0 UNPARSEABLE_RELAY  Informational: message has unparseable relay
lines
 1.3 RCVD_IN_BL_SPAMCOP_NET RBL: Transportiert via Rechner in Liste von
www.spamcop.net
   [Blocked - see
]
 3.3 URIBL_AB_SURBL Enthält URL in AB-Liste (www.surbl.org)
[URIs: dagwizhua -dot- com]
 2.6 URIBL_OB_SURBL Enthält URL in OB-Liste (www.surbl.org)
[URIs: dagwizhua -dot- com]
 3.6 URIBL_SC_SURBL Enthält URL in SC-Liste  (www.surbl.org)
[URIs: dagwizhua -dot- com]
-0.2 AWLAWL: From: address is in the auto white-list

How can the results be so very different on the same spam process? Why would
a few additional headers make a difference if the Bayes does not seem to add
anything to the mail and there is no particular rule for those headers? And
why does a manual scan produce a completely different result if the service
that creates the actual results is the same process?

Thanks for advice
Björn
-- 
View this message in context: 
http://www.nabble.com/Newbie-Questions%3A-Different-Results-for-the-same-message-tp20809927p20809927.html
Sent from the SpamAssassin - Users mailing list archive at Nabble.com.

Re: [sa-list] Re: [sa-list] Re: Spamd and ipv6

[SM]

At 18:23 02-12-2008, Byung-Hee HWANG wrote:
Are you using FreeBSD or NetBSD? If so, i understand you. 
Unfortunately, SA developers do not care about IPv6 yet. So here SA 
program at first do action with "127.0.0.1" than "::1", i guess ;;


This was tested on a BSD system.  SpamAssassin developers are sharing 
their code for free.If we need a specific feature or find a bug, 
we can always send a patch.  If you read the URL I posted previously, 
you will see that the developers have been working on IPv6 support.


Regards,
-sm