Re: Received from rule?

[Mr. Gus]

Christian Nygaard wrote:
How do I write a local spamassassin rule that matches received from 
header mail?.example.com . I've tried
writing a header matching rule but it doesnt seem to work. Can you show 
a short example

of a working header matching rule for received from?

Received: from mail3.example.com 


Something like

header  HappyRuleName   Received =~ /mail\d\.example\.com/i

or

header  HappyRuleName   Received =~ /^from mail\d+\.example\.com/i

?

--
Gus

Re: help with training bayesian filter

[Mr. Gus]

On Wed, Oct 17, 2007 at 04:27:52PM -0700, sinnerman wrote:
> 
> Hi All,
> 
> I currently have SpamAssaassin setup on my FreeBSD machine and have trained
> it with spam and ham messages (greater than the min thresholds of 200/200).
> However, I'm not sure it's setup correctly, nor do I see any obvious results
> (reduced spam) of the training process. A couple of questions:
> 
> * I'm running sa-learn from my own cron (i.e., as my login user), but I'm
> running spamd as "nobody". Since (I believe) the bayesian database is being
> created in my home directory, will spamd be able to access it, or will it
> instead try to access another database? Spamc is also being run from my own
> account. If this setup is not correct, how can I fix it?

I have a systemwide config so I don't know from experience, but are you
running spamd with -x or setting the user with -u? Because if you are, that
might be mucking you up.

> * I cannot get autolearn to work. I've set "bayes_auto_learn_threshold_spam"
> to 8, but even with messages which receive a score greater than 8, the
> message's X-Spam-Status header still says "autolearn=no". Do I need to
> enable autolearning in some other way?

Do you have bayes_auto_learn set? That's what turns it on/off.

-- 
Gus

Re: Bit OT but it's about SPAM

[Mr. Gus]

On Wed, Oct 17, 2007 at 10:46:04AM -0400, Tom Ray wrote:
> I just thought if anyone hasn't read it yet, this article might be 
> interesting to many of you. According to this report SPAM has now 
> reached being 95% of all email.


Made me curious, so I made some stats from my own mail server, just using
data from yesterday...


95.5% spam
 4.5% ham

9,007,657 rejected at MTA (total)
8,798,311 rejected at MTA (not counting SA)
8,390,034 rejected not due to invalid recipients/syntax errors
420,019 scored as spam
210,673 accepted and scored as spam
377,420 accepted and scored as ham
27,579 accepted, but never scored

8600707 is spam (rejected not due etc + accepted and scored spam)
404999 is ham (accepted and scored ham + never scored)
9005706 total messages (above two numbers)


Certain SpamAssassin-related numbers might not add up 100% since I got some
stuff from the SpamAssassin logs, and others from sa-exim's output in exim's
logs, but it should be more or less right. Most of the MTA rejects come
from XBL/SBL/PBL and some in-house blacklists. XBL/SBL/PBL don't even get
to RCPT TO, so perhaps would have been sent to invalid recipients or what
not had we given them the chance.

-- 
Gus

Re: bayes_seen = 256GB

[Mr. Gus]

On Wed, Sep 19, 2007 at 02:11:19PM -0700, mfahey wrote:
> 
> SpamAssassin-3.2.0
> Freebsd6.2
> 
> The file bayes_seen has grown in size to 256GB!  (274992939008)
> How do I cap the size limit of this file? I want to have it not grow larger
> then say 800mb at the most!

You need to expire old bayes tokens. The limit is set not as a size, but as
a count of tokens. The default is 150,000 tokens iirc, but you can set it
yourself by setting the bayes_expiry_max_db_size to whatever value you
want. Pretty much any number you'd reasonably choose will put you less than
800mb. :)

To make it expire, either set bayes_auto_expire to 1 and let it expire
tokens automatically, or run sa-learn --force-expire and sa-learn --sync.

-- 
Gus

Re: Compiling Rules

[Mr. Gus]

Sorry for the dupes. Had a wrong setting in mutt and thought these two 
didn't get sent properly. :-/ (my solution to this problem being to send a
third message... Hmmm...)

-- 
Gus

Display DCC results in headers

[Mr. Gus]

I dunno if anybody else will find this useful, but I made a modification to
DCC.pm that will make it display the same DCC results via the SpamAssassin
report that dcc would normally add in it's header.


http://www.disco-zombie.net/tmp/dcc_header_plugin.tar.gz



Normally, dccproc/ifd/whatever adds a header that looks like this:

X-DCC--Metrics: lum 1049; Body=17 Fuz1=17 Fuz2=17

Done through SA, you just either trip the DCC_CHECK rule or you don't.


My organization is going to be adding dcc to our mail system fairly soon,
and we want people to be able to see the actual results, yet due to our
mail system's current configuration, SpamAssassin is by far the best way to
perform the DCC checks, so I just made SpamAssassin add the info...


The results look like this:

*2 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
*  [lum 1049; Body=1 Fuz1=1 Fuz2=many]

or this:

* -0.0 DCC_CHECK_NEGATIVE Not listed in DCC
*  [lum 1049; Body=16 Fuz1=16 Fuz2=16]


If you only have the report header on when something is scored as spam,
just apply the patch and you're set. Otherwise, just add the following
scoring to one of your config files:

score DCC_CHECK 0 1.37 0 2.17
full DCC_CHECK  eval:check_dcc(0)
describe DCC_CHECK  Listed in DCC 
(http://rhyolite.com/anti-spam/dcc/)
tflags DCC_CHECKnet

full DCC_CHECK_NEGATIVE eval:check_dcc(1)
describe DCC_CHECK_NEGATIVE Not listed in DCC
tflags DCC_CHECK_NEGATIVE   net
score DCC_CHECK_NEGATIVE0 -0.0001 0 -0.0001


And since I'm not a fan of directly modifying my SA installation, I keep
the modified DCC.pm in with my SA config and just specify the path.


Anyway, like I said, dunno if anybody'll find it useful, but since I did it
I figured I might as well share it just in case. I'm sure now that I've
gone to the trouble somebody's going to point me to some "dcc_add_header 1"
config variable that I overlooked. ;^)

-- 
Gus

Re: Compiling Rules

[Mr. Gus]

On Fri, Sep 14, 2007 at 03:54:28PM -0400, Jason Bertoch wrote:
> 
> sa-compile appears to examine rules downloaded via sa-update, including custom
> channels like dostech.  However, it does not appear to pick up custom rules, 
> or
> anything else, from files in /etc/mail/spamassassin.  Is this a bug or a
> feature?

I know for a fact that it picks up my custom rules, as sa-compile couldn't
parse a couple of them correctly. :)

Try running this:

sa-compile -D --list 2>&1 | grep "rules dir"

You should see:

[17010] dbg: config: using "/var/lib/spamassassin/3.002003" for default
rules dir
[17010] dbg: config: using "/etc/mail/spamassassin" for site rules dir


You can also pick a custom body rule that you made and grep that:

sa-compile -D --list 2>&1 | grep TEST

And get results like:

orig TEST_0001 /test01010101/i
r test01010101:TEST_0001

-- 
Gus

Display DCC results in headers

[Mr. Gus]

I dunno if anybody else will find this useful, but I made a modification to
DCC.pm that will make it display the same DCC results via the SpamAssassin
report that dcc would normally add in it's header.


http://www.disco-zombie.net/tmp/dcc_header_plugin.tar.gz



Normally, dccproc/ifd/whatever adds a header that looks like this:

X-DCC--Metrics: lum 1049; Body=17 Fuz1=17 Fuz2=17

Done through SA, you just either trip the DCC_CHECK rule or you don't.


My organization is going to be adding dcc to our mail system fairly soon,
and we want people to be able to see the actual results, yet due to our
mail system's current configuration, SpamAssassin is by far the best way to
perform the DCC checks, so I just made SpamAssassin add the info...


The results look like this:

*2 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
*  [lum 1049; Body=1 Fuz1=1 Fuz2=many]

or this:

* -0.0 DCC_CHECK_NEGATIVE Not listed in DCC
*  [lum 1049; Body=16 Fuz1=16 Fuz2=16]


If you only have the report header on when something is scored as spam,
just apply the patch and you're set. Otherwise, just add the following
scoring to one of your config files:

score DCC_CHECK 0 1.37 0 2.17
full DCC_CHECK  eval:check_dcc(0)
describe DCC_CHECK  Listed in DCC 
(http://rhyolite.com/anti-spam/dcc/)
tflags DCC_CHECKnet

full DCC_CHECK_NEGATIVE eval:check_dcc(1)
describe DCC_CHECK_NEGATIVE Not listed in DCC
tflags DCC_CHECK_NEGATIVE   net
score DCC_CHECK_NEGATIVE0 -0.0001 0 -0.0001


And since I'm not a fan of directly modifying my SA installation, I keep
the modified DCC.pm in with my SA config and just specify the path.


Anyway, like I said, dunno if anybody'll find it useful, but since I did it
I figured I might as well share it just in case. I'm sure now that I've
gone to the trouble somebody's going to point me to some "dcc_add_header 1"
config variable that I overlooked. ;^)

-- 
Gus

Re: Compiling Rules

[Mr. Gus]

On Fri, Sep 14, 2007 at 03:54:28PM -0400, Jason Bertoch wrote:
> 
> sa-compile appears to examine rules downloaded via sa-update, including custom
> channels like dostech.  However, it does not appear to pick up custom rules, 
> or
> anything else, from files in /etc/mail/spamassassin.  Is this a bug or a
> feature?

I know for a fact that it picks up my custom rules, as sa-compile couldn't
parse a couple of them correctly. :)

Try running this:

sa-compile -D --list 2>&1 | grep "rules dir"

You should see:

[17010] dbg: config: using "/var/lib/spamassassin/3.002003" for default
rules dir
[17010] dbg: config: using "/etc/mail/spamassassin" for site rules dir


You can also pick a custom body rule that you made and grep that:

sa-compile -D --list 2>&1 | grep TEST

And get results like:

orig TEST_0001 /test01010101/i
r test01010101:TEST_0001

-- 
Gus

Re: [OT] Seeing increase in smtp concurrency ?

[Mr. Gus]

Rajkumar S wrote:

Hi,

Does any one seeing increasing smtp concurrency for the past couple of
weeks? I run couple of (qmail/simscan/spamassassin) mail servers and
all experience the same problem. The spam does not increase, but this
is hogging my mail servers. Probably a new crop of spamming tools?

I am attaching one qmail-mtrg graph that shows the problem.

http://img403.imageshack.us/img403/2224/smtpmonthyq4.png


Yeah, two weekends ago our mail servers got absolutely slammed with 
connections that were left open. They'd get rejected, and then leave the 
connection open for a while, then try again, and so on.


Our count of exim processes per server went up from our already 
higher-than-it-recently-used-to-be 500 to around 1,000. This continued 
on monday, climbing to around 1300. Restarting exim and killing off 
these old connections would only relieve it for ten minutes or less. 
When it hit 1500, we moved a few of our rbls to the connect phase, which 
brought it down to about 150 exim procs. Yeesh...


--
Gus