Re: Spam tagging not happening
Dominic, Ok it seems like SAEximDebug is set to 1, but I don't see anything similar like in your example log My /var/log/exim4/mainlog : 2010-10-19 08:13:50 1P85Q4-0003iH-Gw = p...@decordeli.com H= wblv-ip-mesg-2-3.saix.net [196.25.240.101] P=esmtp S=26973403 id= c865a021-d0c2-4859-a538-ae67b5626...@decordeli.com 2010-10-19 08:13:52 1P85Q4-0003iH-Gw = paul p...@mydomain.com R=local_user T=maildir_home 2010-10-19 08:13:52 1P85Q4-0003iH-Gw Completed Surely I'm missing something here and when I do a grep sa-exim /var/lib/exim4/config.autogenerated, the output is null. Does this mean I don't have sa-exim configured properly? I really appreciate the help guys:-) Jerry On Mon, Oct 18, 2010 at 5:48 PM, Dominic Benson domi...@lenny.cus.orgwrote: On 18/10/10 16:11, Jeremy Van Rooyen wrote: Thanks for the quick reply Dominic, I just checked and the SApermreject is set sensible for now. The latter part of your email refers to SA-Flagged messages, how do I make sure this is working, as I have enabled rewrite_header in /etc/spamassassin/local.cf. http://local.cf/ If rewite_header is enabled, and you don't see the *SPAM* (or alternative you specified) in the subject line, then it didn't get processed. Could you set SAEximDebug: 1 in /etc/exim4/sa-exim.conf, and then paste the output of a message in /var/log/exim4/mainlog, e.g.: 2010-10-18 16:26:48 1P7rbr-0007e3-NS SA: Debug: SAEximRunCond expand returned: 'true' 2010-10-18 16:26:48 1P7rbr-0007e3-NS SA: Debug: check succeeded, running spamc 2010-10-18 16:26:51 1P7rbr-0007e3-NS SA: Action: permanently rejected message: score=15.7 required=5.0 trigger=12.0 (scanned in 3/3 secs | Message-Id: 1P7rbr-0007e3-NS). From e...@cambridge-union.orge...@cambridge-union.org(host=NULL [117.5.37.103]) for e...@cambridge-union.org Then we should be able to see what is actually happening. Also, could you check the output of grep sa-exim /var/lib/exim4/config.autogenerated - if you use Exim in unsplit configuration, sa-exim doesn't get used by default. exim4-daemon-heavy has an alternative way of using SpamAssassin (exiscan_acl), which is powerful, but not so convenient. How do I add a message rule that subject starts with *SPAM* ? Do I add to my local.cf? I'm sure I did this already. I mean you should do this in either your mail client, or e.g. a Cyrus sieve. This is about using the data that SpamAssassin/Exim have added to the message to classify it in your inbox, rather than a mail routing decision. (e.g. in Thunderbird you would go Tools - Message Filters - New) Dominic -- There is therefore now no condemnation to them which are in Christ Jesus, who walk not after the flesh, but after the Spirit. Romans 8
Re:Spam tagging not happening
Any news for me on this issue? Dominic, Ok it seems like SAEximDebug is set to 1, but I don't see anything similar like in your example log My /var/log/exim4/mainlog : 2010-10-19 08:13:50 1P85Q4-0003iH-Gw = p...@decordeli.com H= wblv-ip-mesg-2-3.saix.net [196.25.240.101] P=esmtp S=26973403 id= c865a021-d0c2-4859-a538-ae67b5626...@decordeli.com 2010-10-19 08:13:52 1P85Q4-0003iH-Gw = paul p...@mydomain.com R=local_user T=maildir_home 2010-10-19 08:13:52 1P85Q4-0003iH-Gw Completed Surely I'm missing something here and when I do a grep sa-exim /var/lib/exim4/config.autogenerated, the output is null. Does this mean I don't have sa-exim configured properly? I really appreciate the help guys:-) Jerry On Mon, Oct 18, 2010 at 5:48 PM, Dominic Benson domi...@lenny.cus.orgwrote: On 18/10/10 16:11, Jeremy Van Rooyen wrote: Thanks for the quick reply Dominic, I just checked and the SApermreject is set sensible for now. The latter part of your email refers to SA-Flagged messages, how do I make sure this is working, as I have enabled rewrite_header in /etc/spamassassin/local.cf. http://local.cf/ If rewite_header is enabled, and you don't see the *SPAM* (or alternative you specified) in the subject line, then it didn't get processed. Could you set SAEximDebug: 1 in /etc/exim4/sa-exim.conf, and then paste the output of a message in /var/log/exim4/mainlog, e.g.: 2010-10-18 16:26:48 1P7rbr-0007e3-NS SA: Debug: SAEximRunCond expand returned: 'true' 2010-10-18 16:26:48 1P7rbr-0007e3-NS SA: Debug: check succeeded, running spamc 2010-10-18 16:26:51 1P7rbr-0007e3-NS SA: Action: permanently rejected message: score=15.7 required=5.0 trigger=12.0 (scanned in 3/3 secs | Message-Id: 1P7rbr-0007e3-NS). From e...@cambridge-union.orge...@cambridge-union.org(host=NULL [117.5.37.103]) for e...@cambridge-union.org Then we should be able to see what is actually happening. Also, could you check the output of grep sa-exim /var/lib/exim4/config.autogenerated - if you use Exim in unsplit configuration, sa-exim doesn't get used by default. exim4-daemon-heavy has an alternative way of using SpamAssassin (exiscan_acl), which is powerful, but not so convenient. How do I add a message rule that subject starts with *SPAM* ? Do I add to my local.cf? I'm sure I did this already. I mean you should do this in either your mail client, or e.g. a Cyrus sieve. This is about using the data that SpamAssassin/Exim have added to the message to classify it in your inbox, rather than a mail routing decision. (e.g. in Thunderbird you would go Tools - Message Filters - New) Dominic -- There is therefore now no condemnation to them which are in Christ Jesus, who walk not after the flesh, but after the Spirit. Romans 8 -- There is therefore now no condemnation to them which are in Christ Jesus, who walk not after the flesh, but after the Spirit. Romans 8
Re: Spam tagging not happening
Surely I'm missing something here and when I do a grep sa-exim /var/lib/exim4/config.autogenerated, the output is null. Does this mean I don't have sa-exim configured properly? It means that it isn't being used by exim. We're veering away from SA-Users topics, but: if you dpkg-reconfigure exim4-config and select Yes to the question Split configuration into small files, then you should find that SA-Exim is used; it [sa-exim] installs a config file at /etc/exim4/conf.d/main/15_sa-exim_plugin_path - the /etc/exim4/conf.d directory is what gets compiled [by the exim4 init script] into /var/lib/exim4/config.autogenerated if you select the small files config method. Otherwise it uses the monolithic config template at /etc/exim4/exim4.conf.template - which doesn't get the SA-Exim stuff added automatically. I really appreciate the help guys:-) Jerry Dominic
Bayes timeouts and database handle being DESTROY'd without explicit disconnect
Hello, I'm running a busy mail server. We've got a bayes database on its own server, with InnoDB tables. I'm seeing a number of these entries in my log files and am struggling to determine what could be causing them and how to fix them: Oct 19 07:02:10 spamd3 spamd[27474]: learn: exceeded time limit in pms learn Oct 17 06:30:12 spamd3 spamd[25651]: plugin: eval failed: bayes: (in learn) __alarm__ignore__(15190) Oct 17 06:30:42 spamd3 spamd[25598]: plugin: eval failed: bayes: (in learn) child processing timeout at /usr/sbin/spamd line 1283, GEN1295 line 185. I get quite a few of these: Oct 19 07:02:19 spamd3 spamd[18746]: Issuing rollback() for database handle being DESTROY'd without explicit disconnect() at /usr/share/perl5/Mail/SpamAssassin/Plugin/Bayes.pm line 1516, GEN19133 line 2. and a few of these, although not that many: Oct 17 12:02:29 spamd3 spamd[6367]: prepare_cached(SELECT max(runtime) from bayes_expire WHERE id = ?) statement handle DBI::st=HASH(0xadbb060)still Active at /usr/share/perl5/Mail/SpamAssassin/BayesStore/SQL.pm line 722 Oct 19 05:33:13 spamd3 spamd[1630]: bayes: db_seen corrupt: value='1287482415' for 5d6fb52248450ee7528848c3a78b5a0650a24...@sa_generated, ignored at /usr/share/perl5/Mail/SpamAssassin/Plugin/Bayes.pm line 397, GEN18675 line 112. thanks for any insights! micha pgpOWKtRHjXPz.pgp Description: PGP signature
Re: Bayes timeouts and database handle being DESTROY'd without explicit disconnect
On 19 Oct 2010, at 17:05, Micah Anderson wrote: Hello, I'm running a busy mail server. We've got a bayes database on its own server, with InnoDB tables. What is your total DB size / server RAM? Could you include a snapshot of the output of top from the DB server? I would guess that your problem is indexing/tuning or server capacity MySQL side rather than in SA, but without more data it is just a guess. I'm seeing a number of these entries in my log files and am struggling to determine what could be causing them and how to fix them: Oct 19 07:02:10 spamd3 spamd[27474]: learn: exceeded time limit in pms learn Oct 17 06:30:12 spamd3 spamd[25651]: plugin: eval failed: bayes: (in learn) __alarm__ignore__(15190) Oct 17 06:30:42 spamd3 spamd[25598]: plugin: eval failed: bayes: (in learn) child processing timeout at /usr/sbin/spamd line 1283, GEN1295 line 185. I get quite a few of these: Oct 19 07:02:19 spamd3 spamd[18746]: Issuing rollback() for database handle being DESTROY'd without explicit disconnect() at /usr/share/perl5/Mail/SpamAssassin/Plugin/Bayes.pm line 1516, GEN19133 line 2. If the processing timeout above occurs inside an uncommitted transaction, you will get this. and a few of these, although not that many: Oct 17 12:02:29 spamd3 spamd[6367]: prepare_cached(SELECT max(runtime) from bayes_expire WHERE id = ?) statement handle DBI::st=HASH(0xadbb060)still Active at /usr/share/perl5/Mail/SpamAssassin/BayesStore/SQL.pm line 722 Try an EXPLAIN SELECT max(runtime) from bayes_expire WHERE id = some value; as you know it to be slow it might give a clue where to look to improve performance. Or try turning the general query log on for a while and see what queries are taking up time. MonYog is quite a nice frontend to this, but you can do it by hand fairly simply. Oct 19 05:33:13 spamd3 spamd[1630]: bayes: db_seen corrupt: value='1287482415' for 5d6fb52248450ee7528848c3a78b5a0650a24...@sa_generated, ignored at /usr/share/perl5/Mail/SpamAssassin/Plugin/Bayes.pm line 397, GEN18675 line 112. thanks for any insights! micha Dominic
Spam US$350,000 not tripped
I am surprised this plain text spam did not trip for US$350,000 sa 3.2.4 http://www.Real-World-Systems.com/mail/spam.un
Re: Spam US$350,000 not tripped
On 19/10/10 22:34, Dennis German wrote: I am surprised this plain text spam did not trip for US$350,000 sa 3.2.4 http://www.Real-World-Systems.com/mail/spam.un It hits a stack of rules here (some are my own scoring) - looks like it's time to upgrade to SA 3.3.1. X-Spam-Report: * 6.0 BAYES_99 BODY: Bayes spam probability is 99 to 100% * [score: 0.] * 25 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT * [148.208.170.3 listed in bb.barracudacentral.org] * 3.0 RCVD_IN_JMF_BL RBL: Relay listed in JunkEmailFilter BLACK (bad) * [148.208.170.3 listed in hostkarma.junkemailfilter.com] * 1.8 DKIM_ADSP_DISCARD No valid author signature, domain signs all mail * and suggests discarding the rest * 1.0 MISSING_HEADERS Missing To: header * 0.0 T_LOTS_OF_MONEY Huge... sums of money * 1.6 REPLYTO_WITHOUT_TO_CC REPLYTO_WITHOUT_TO_CC * 3.0 JM_SOUGHT_FRAUD_3 Body contains frequently-spammed text patterns * 3.4 FILL_THIS_FORM_LONG Fill in a form with personal information * 0.0 T_FILL_THIS_FORM Fill in a form with personal information * 1.5 ADVANCE_FEE_4_NEW Appears to be advance fee fraud (Nigerian 419) * 3.3 ADVANCE_FEE_5_NEW Appears to be advance fee fraud (Nigerian 419) * 0.5 ADVANCE_FEE_3_NEW Appears to be advance fee fraud (Nigerian 419) * 0.5 ADVANCE_FEE_2_NEW_MONEY Advance Fee fraud and lots of money * 0.9 ADVANCE_FEE_3_NEW_FORM Advance Fee fraud and a form * 1.0 ADVANCE_FEE_3_NEW_MONEY Advance Fee fraud and lots of money * 0.8 ADVANCE_FEE_2_NEW_FORM Advance Fee fraud and a form * 0.5 MONEY_FRAUD_5 Lots of money and many fraud phrases * 0.8 MONEY_FRAUD_8 Lots of money and very many fraud phrases * 0.5 MONEY_FRAUD_3 Lots of money and several fraud phrases * 0.5 FORM_FRAUD_5 Fill a form and many fraud phrases * 0.5 FORM_FRAUD_3 Fill a form and several fraud phrases
Re: Spam US$350,000 not tripped
On Tue, 2010-10-19 at 22:41 +0100, Ned Slider wrote:
On 19/10/10 22:34, Dennis German wrote:
I am surprised this plain text spam did not trip for US$350,000
sa 3.2.4
Uhm, a generic amount of money on it's own is not a sign of spam. You
know, some people do deal with and talk about money...
It hits a stack of rules here (some are my own scoring) - looks like
it's time to upgrade to SA 3.3.1.
* 6.0 BAYES_99 BODY: Bayes spam probability is 99 to 100%
* [score: 0.]
* 25 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT
* [148.208.170.3 listed in bb.barracudacentral.org]
Seriously? Or is that a score typo in your cf files?
* 3.0 RCVD_IN_JMF_BL RBL: Relay listed in JunkEmailFilter BLACK
(bad)
* [148.208.170.3 listed in hostkarma.junkemailfilter.com]
BRBL and JMF are easy enough to add to an existing 3.2.x installation.
* 1.0 MISSING_HEADERS Missing To: header
Stock 3.2.x, scored even slightly higher.
* 3.0 JM_SOUGHT_FRAUD_3 Body contains frequently-spammed text
patterns
Easy enough to add to 3.2.x via sa-update. Recommended.
Bayes of course also is part of stock 3.2.x. ;) Plethora of new fraud
rules snipped.
--
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Spam tagging not happening
Hi Dominic and Users, I was not using the split configuration of exim4, I'm using the monolithic config at /etc/exim4/exim4.conf.template. So I added this line to my /etc/exim4/exim4.conf.template config file right at the top local_scan_path = /usr/lib/exim4/local_scan/sa-exim.so restarted exim4 and I'm seeing SA entries in my /var/log/exim4/main.log. I then did a grep sa-exim /var/lib/exim4/config.autogenerated and the results obviously was local_scan_path = /usr/lib/exim4/local_scan/sa-exim.so. I really appreciate the help here guys, and I'm very happy that I'm one step forward now. Thanks Jerry On Tue, Oct 19, 2010 at 4:10 PM, Dominic Benson domi...@lenny.cus.orgwrote: Surely I'm missing something here and when I do a grep sa-exim /var/lib/exim4/config.autogenerated, the output is null. Does this mean I don't have sa-exim configured properly? It means that it isn't being used by exim. We're veering away from SA-Users topics, but: if you dpkg-reconfigure exim4-config and select Yes to the question Split configuration into small files, then you should find that SA-Exim is used; it [sa-exim] installs a config file at /etc/exim4/conf.d/main/15_sa-exim_plugin_path - the /etc/exim4/conf.d directory is what gets compiled [by the exim4 init script] into /var/lib/exim4/config.autogenerated if you select the small files config method. Otherwise it uses the monolithic config template at /etc/exim4/exim4.conf.template - which doesn't get the SA-Exim stuff added automatically. I really appreciate the help guys:-) Jerry Dominic -- There is therefore now no condemnation to them which are in Christ Jesus, who walk not after the flesh, but after the Spirit. Romans 8
Re: Spam US$350,000 not tripped
On Oct 19, 2010, at 5:56 PM, Karsten Bräckelmann wrote: On Tue, 2010-10-19 at 22:41 +0100, Ned Slider wrote: On 19/10/10 22:34, Dennis German wrote: I am surprised this plain text spam did not trip for US$350,000 sa 3.2.4 Uhm, a generic amount of money on it's own is not a sign of spam. You know, some people do deal with and talk about money... It hits a stack of rules here (some are my own scoring) - looks like it's time to upgrade to SA 3.3.1. * 6.0 BAYES_99 BODY: Bayes spam probability is 99 to 100% * [score: 0.] * 25 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT * [148.208.170.3 listed in bb.barracudacentral.org] Seriously? Or is that a score typo in your cf files? * 3.0 RCVD_IN_JMF_BL RBL: Relay listed in JunkEmailFilter BLACK (bad) * [148.208.170.3 listed in hostkarma.junkemailfilter.com] BRBL and JMF are easy enough to add to an existing 3.2.x installation. * 1.0 MISSING_HEADERS Missing To: header Stock 3.2.x, scored even slightly higher. * 3.0 JM_SOUGHT_FRAUD_3 Body contains frequently-spammed text patterns Easy enough to add to 3.2.x via sa-update. Recommended. Bayes of course also is part of stock 3.2.x. ;) Plethora of new fraud rules snipped. Karsten, Thank you fro the suggestion of adding BRBL and JMF. Can you please point me to some detailed information explaining how to do that. PS I am on a shared server without root access. ( or I would have upgraded SA)
Re: Spam US$350,000 not tripped
On Tue, 2010-10-19 at 19:29 -0400, Dennis German wrote:
Thank you fro the suggestion of adding BRBL and JMF.
Can you please point me to some detailed information explaining how to do
that.
PS I am on a shared server without root access. ( or I would have upgraded SA)
The actual rules to be added are documented in SA bugzilla. The Sought
channel is documented in the wiki.
However, no root access -- neither of these are user preferences, it is
impossible to add with mere tweaking of user_prefs [1]. You can only do
this, if you have access to the site-wide config, commonly referred to
as local.cf.
This might be possible, even on a shared, virtual server. If you ever
could add rules yourself, you can do this, too.
[1] Unless allow_user_rules is enabled, which is rather unlikely.
--
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Spam US$350,000 not tripped
On 19/10/10 22:56, Karsten Bräckelmann wrote: On Tue, 2010-10-19 at 22:41 +0100, Ned Slider wrote: On 19/10/10 22:34, Dennis German wrote: I am surprised this plain text spam did not trip for US$350,000 sa 3.2.4 Uhm, a generic amount of money on it's own is not a sign of spam. You know, some people do deal with and talk about money... It hits a stack of rules here (some are my own scoring) - looks like it's time to upgrade to SA 3.3.1. * 6.0 BAYES_99 BODY: Bayes spam probability is 99 to 100% * [score: 0.] * 25 RCVD_IN_BRBL_LASTEXT RBL: RCVD_IN_BRBL_LASTEXT * [148.208.170.3 listed in bb.barracudacentral.org] Seriously? Or is that a score typo in your cf files? I did say above some are my own scoring. I've been evaluating BRBL to see if it's a candidate to use at the smtp level and need to identify possible false positives. Giving it a ridiculously high score ensures any hits end up in quarantine where I can examine. No FPs of note yet. I've also tweaked the Basian scoring for my own preferences. I still see a fair amount of spam caught by Bayes alone and manually train Bayes with confirmed ham/spam only. I have high confidence in my Bayesian setup and whitelisting invariably catches any potential FP hits. In general, I wouldn't recommend users tweak the default scoring too much.
