Re: Spamassassin Timeout error
On Fri, 2014-02-21 at 14:57 -0800, John Hardin wrote: > On Sat, 22 Feb 2014, parakrama55 . wrote: > > > 3 .Do you know how to change the timeout in the SA glue? > > No , please advice > > > > 4. What is your SA glue layer? > > > > Both exim and spamassin services runs in the same server , exim calls > > spamassin via 127.0.0.1 address > > This may help: > http://commons.oreilly.com/wiki/index.php/SpamAssassin/Integrating_SpamAssassin_with_Exim#Setting_a_timeout_on_spamc > I think that is a bit old; not sure if sa-Exim is still supported/maintained. This link may be more useful: http://www.exim.org/exim-html-current/doc/html/spec_html/ch-content_scanning_at_acl_time.html Typically we limit the resources SA uses by message size (done by Exim), and set a timeout for SA (done by SA using 'time_limit' in our local.cf file). John. -- John Horne Tel: +44 (0)1752 587287 Plymouth University, UK Fax: +44 (0)1752 587001
Re: Spamassassin Timeout error
On Fri, 2014-02-21 at 19:24 +0530, parakrama55 . wrote: > we are getting follwing errors in the logs, > > 2014-02-21 05:50:27 1WGqSi-0003d7-Pi spam acl condition: error reading > from spamd socket: Connection timed out > 2014-02-21 05:50:27 1WGqSi-0003dD-9p spam acl condition: error reading > from spamd socket: Connection timed out > Hello, We get quite a few of these on a couple of our old mail servers. They are overloaded systems, and so it takes too long to run SA. Exim cuts in in order for the message to actually be accepted within an overall 5 minute time limit. You might want to check the SA and exim logs to see what is going on, and run 'top' to see how busy the servers get. John. -- John Horne Tel: +44 (0)1752 587287 Plymouth University, UK Fax: +44 (0)1752 587001
Re: Rules not working
On Mon, 2013-09-09 at 02:19 +, Raymond Jette wrote: > Thanks for the information. When running it this way everything > works. I'm not sure why it is not working with normal mail flow. > I don't think you mentioned which O/S you are using. However, you may (it depends on your O/S) find spamassassin startup options in the '/etc/sysconfig/spamassassin' file. As you are using exim you could run exim itself in debug mode to maybe see what is happening. Try something like: exim -d'+all' recipient_addr /tmp/exim-test 2>&1 where 'recipient_addr' is some suitable recipient address for your site. This will produce a lot of output, hence it is redirected to the file '/tmp/exim-test'. The file '/tmp/msg' is an email message, complete with headers. You should then be able to look in the '/tmp/exim-test' file and see the flow of the message through exim. In particular what happens to it when SA is called (if it is called). (I assume you are using 'spam' in some ACL to call spamassassin.) John. -- John Horne Tel: +44 (0)1752 587287 Plymouth University, UK Fax: +44 (0)1752 587001
Re: Using sa-compile are local rules compiled?
On Thu, 2013-04-18 at 14:08 +0100, RW wrote: > On Wed, 17 Apr 2013 21:40:33 +0100 > John Horne wrote: > > > Hello, > > > > We are running SpamAssassin 3.3.2 on a CentOS 5.9 server. sa-update > > runs via a daily cron job, and we have modified that to run > > sa-compile as well. However, there are some questions: > > > > sa-compile is run without any options. So what I am unsure of is > > whether our local rules (in /etc/mail/spamassassin/local-spam.cf) are > > compiled as well or not? > > They are for me, but it's easy enough to test. Find a few *simple* > local body rules and grep for them under the "compiled" directory. > > > The man page for sa-compile says that the '--siteconfigpath' option > > defaults to /etc/mail/spamassassin which I assume implies that our > > local rules would be compiled? > > > > If they are, and we want to change our local rules, then I assume we > > would have to re-compile all the rules before restarting SpamAssassin? > > That's the received wisdom, but I've never seen a definitive reason > why. Compiled rules are intended to co-exit with non-compiled rules, > and from my limited testing, rules behave correctly when they are added, > removed or modified without recompiling. However I have seen one case > where a rule wasn't working properly and was apparently fixed by a > recompile, but I suspect that was a specific bug. > Thanks for the reply. I (now) gather that compilation actually only applies to 'body' rules. We don't have too many of those, so generally for non-body local rules a simple restart of SA works immediately. I suspect that for the body rules we never noticed that they only started to be hit the following day (recompiling of rules occurs each night). We would have simply assumed that the rule didn't have any hits until that time. It is, of course, something we will now bear in mind :-) John. -- John Horne, Plymouth University, UK Tel: +44 (0)1752 587287Fax: +44 (0)1752 587001
Using sa-compile are local rules compiled?
Hello, We are running SpamAssassin 3.3.2 on a CentOS 5.9 server. sa-update runs via a daily cron job, and we have modified that to run sa-compile as well. However, there are some questions: sa-compile is run without any options. So what I am unsure of is whether our local rules (in /etc/mail/spamassassin/local-spam.cf) are compiled as well or not? The man page for sa-compile says that the '--siteconfigpath' option defaults to /etc/mail/spamassassin which I assume implies that our local rules would be compiled? If they are, and we want to change our local rules, then I assume we would have to re-compile all the rules before restarting SpamAssassin? Thanks, John. -- John Horne, Plymouth University, UK Tel: +44 (0)1752 587287Fax: +44 (0)1752 587001
Checking Received headers
Hello, Using SA 3.3.1 can I ask how the 'header' command in a rule treats the Received: headers? For example, if I have: header LOCAL_HDR_CHECK Received =~ / from \S+\.plymouth\.ac\.uk / Does SA concatenate all the Received headers together, and then check the regex against that? Thanks, John. -- John Horne Tel: +44 (0)1752 587287 University of Plymouth, UK Fax: +44 (0)1752 587001
Re: New plugin: DecodeShortURLs
On Mon, 2010-10-04 at 22:55 +0100, John Horne wrote: > > I grabbed a copy of the above plugin and tried it this afternoon (on a > CentOS 5.5 system). We log all our spamd messages to /var/log/maillog > via syslog. For the plugin I disabled all the options except > 'url_shortener_syslog' which was set to 1. > > After restarting SpamAssassin we started to get some messages from spamd > sent to /var/log/mailog and some sent to /var/log/messages. > Hello, Well I suspect the problem is with the Sys::Syslog perl module. On our CentOS 5.5 system we have perl 5.8 with version 0.13 of the module (this is quite old). My Fedora 13 PC uses perl 5.10 with version 0.27 of the module (the latest version). However, it seems there is a bug with that version which causes it to ignore the facility - (fix here) http://rt.cpan.org/Public/Bug/Display.html?id=55151 I have left the plugin enabled, but without using the syslog options. I have had a look at the (0.13) syslog module, but can't really see where the problem is. If I get more time, then I may try and debug it further. John. -- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287Fax: +44 (0)1752 587001
Re: New plugin: DecodeShortURLs
On Thu, 2010-09-23 at 11:30 +0100, Steve Freegard wrote: > > > > Hopefully it will be useful to others; you can grab it from: > > > > http://www.fsl.com/support/DecodeShortURLs.pm > > http://www.fsl.com/support/DecodeShortURLs.cf > > > ... > > - Added option to allow logging to syslog (mail.info). > Hello, I grabbed a copy of the above plugin and tried it this afternoon (on a CentOS 5.5 system). We log all our spamd messages to /var/log/maillog via syslog. For the plugin I disabled all the options except 'url_shortener_syslog' which was set to 1. After restarting SpamAssassin we started to get some messages from spamd sent to /var/log/mailog and some sent to /var/log/messages. Not messages from the plugin, but any messages from spamd. For example (from /var/log/messages): Oct 4 22:28:50 pat sauser[31061]: spamd: checking message <79d9f28c0f1f811a22d92293e4e41...@www.facebook.com> for sauser:10001 Oct 4 22:28:56 pat sauser[31061]: spamd: clean message (-0.1/8.0) for sauser:10001 in 5.6 seconds, 7896 bytes. Oct 4 22:28:56 pat sauser[31061]: spamd: result: . 0 - BAYES_00,DCC_CHECK,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU, HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS scantime=5.6,size=7896,user=sauser,uid=10001,required_score=8.0, rhost=localhost.localdomain,raddr=127.0.0.1,rport=38700, mid=<79d9f28c0f1f811a22d92293e4e41...@www.facebook.com>, bayes=0.00,autolearn=no The messages are not just being duplicated in both files, there are different messages in each file. Our syslog.conf specifies: *.info;mail.none /var/log/messages mail.* -/var/log/maillog I tried changing DecodeShortURLs.pm calls to syslog to use 'info|mail' and that made no difference. I also tried commenting out the 'syslog' calls, and used backtick calls to '/usr/bin/logger' instead. The same problem happened. If I take the plugin out, then all messages from spamd go to /var/log/maillog as before. Anyone any ideas as to what is going on? Thanks, John. -- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287Fax: +44 (0)1752 587001
Re: Problem matching newline in body
On Fri, 2010-05-21 at 06:53 -0700, John Hardin wrote:
> On Fri, 21 May 2010, John Horne wrote:
>
> > Hello,
> >
> > Can you tell it's Friday afternoon? What should be a simple problem
> > always seems to become a nightmare on Friday afternoons! :-)
> >
> > Using SA 3.3.1 I have the following simple rule:
> >
> > body LOCAL_JH /userid:\s*\n/i
> >
> > which should look for 'userid:', any number of spaces and then a NL
> > character (that is, there is nothing following the spaces on the same
> > line).
> >
> > If I send a message containing:
> >
> >some text
> >userid:
> >some more text
>
> The "body" rule processing collapses that paragraph into a single string,
>
Ah, okay that would make sense. It would also explain why the use of the
'm' modifier didn't seem to work either (it would be matching at the
very end of the message).
>
> Try this:
>
> rawbody LOCAL_JH /userid:\s*$/i
>
That failed as-is, but by including the 'm' modifier it works fine :-)
I (now) notice that in the Mail::SpamAssassin::Conf man page it states
for 'rawbody':
...but HTML tags and line breaks will still be present.
Multiline expressions will need to be used to match strings that
are broken by line breaks.
Many thanks,
John.
--
John Horne Tel: +44 (0)1752 587287
University of Plymouth, UK Fax: +44 (0)1752 587001
Re: Problem matching newline in body
On Fri, 2010-05-21 at 15:51 +0200, Yet Another Ninja wrote: > On 2010-05-21 15:40, John Horne wrote: > If I send a message containing: > > some text > userid: > some more text ... > > > > Can someone show me how to match a newline character in the above rule > > please? > > can you post a spam sample @ pastebin? > No spam sample. This is just a simple test of matching a newline. I tested it by sending a plain-text message containing literally the text quoted above. John. -- John Horne Tel: +44 (0)1752 587287 University of Plymouth, UK Fax: +44 (0)1752 587001
Problem matching newline in body
Hello, Can you tell it's Friday afternoon? What should be a simple problem always seems to become a nightmare on Friday afternoons! :-) Using SA 3.3.1 I have the following simple rule: body LOCAL_JH /userid:\s*\n/i which should look for 'userid:', any number of spaces and then a NL character (that is, there is nothing following the spaces on the same line). If I send a message containing: some text userid: some more text it fails. If I insert a NL before 'some more text', then it works. I tried using '/userid:\s*$/mi', but that too didn't work. Can someone show me how to match a newline character in the above rule please? John. -- John Horne Tel: +44 (0)1752 587287 University of Plymouth, UK Fax: +44 (0)1752 587001
Re: Yahoo/URL spam
On Tue, 2010-03-23 at 13:18 -0400, Alex wrote:
> Hi Charles,
>
> >> /^[^a-z]{0,10}(http:\/\/|www\.)(\w+\.)+(com|net|org|biz|cn|ru)\/?[^
> >> ]{0,20}[^a-z]{0,10}$/msi
>
> This is what I have:
>
> /^[^a-z]{0,10}(http:\/\/|www\.)(\w+\.)+(com|net|org|biz|cn|ru)\/?[^
> ]{0,20}[a-z]{0,10}$/msi
^
The original had [^a-z]
John.
--
John Horne, University of Plymouth, UK
Tel: +44 (0)1752 587287Fax: +44 (0)1752 587001
Re: spams to abuse@ id
On Tue, 2009-08-25 at 11:28 +0530, ram wrote: > I am getting a lot of pill spams on the abuse@ ids > > I had thought spammers would not really be that naive. Usually anyone > sitting at the abuse@ helpdesk is atleast smart enough to know not to > respond to these fakes > > They are just creating a datafeed for my blacklists and uri-lists > > Only thing is that the real purpose of having an un-filtered abuse > address is getting defeated if overwhelmed with spams > We get loads of spam at our abuse and postmaster addresses. However, we use SA to score them, and our mail client (evolution) filters them into separate folders depending on how 'spammy' they are. As such most genuine mail is in the main inbox, all other mail is in one of the 'spam' folders (we only have 2 anyway; those scoring 8->18, and those scoring over 18). It makes it manageable. John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: john.ho...@plymouth.ac.uk Fax: +44 (0)1752 587001
Re: Malware list Q
> On Fri, Jul 24, 2009 at 10:34, Brent Clark wrote: > > Do any of you guys use the following list. > > http://malware.hiperlinks.com.br/cgi/submit?action=list_sa > > If so, may I ask how do you find the results, and is it worth adding to > spamassassin. > Hi, We use malwarepatrol with our central squid web caches. Not sure about effectiveness of it though, really should dig out some stats for it perhaps! John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: john.ho...@plymouth.ac.uk Fax: +44 (0)1752 587001
Re: SA report header added to ham mail
On Wed, 2009-07-01 at 01:14 +0200, Karsten Bräckelmann wrote: > On Tue, 2009-06-30 at 21:57 +0100, John Horne wrote: > > However, as far as I can tell, the X-Spam-Report header gets added to > > ham mail as well as spam. For example: > > > >X-spam-report: Score=-6.9 > > tests=BAYES_00,DCC_CHECK,RCVD_IN_DNSWL_HI autolearn=ham > > That is not a standard SA header. Actually, there's quite a lot fishy > about that. > Sorry, lack of information and understanding from my part. SA is called at SMTP time by the Exim MTA. I have now found out that exim invokes a connection to spamd, and hence gets the 'report' back regardless of whether the message is spam or not. Exim then builds up what it calls: $spam_report A multiline text table, containing the full SpamAssassin report for the message. I took the $spam_report variable contents to be the same as the X-Spam-Report header from SA. It is not, it is built from the output received from spamd. (The header quoted above by me is deliberately built by us in the MTA, and called X-Spam-Report by us. Either exim or my mail client is lowercasing part of it.) Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: john.ho...@plymouth.ac.uk Fax: +44 (0)1752 587001
SA report header added to ham mail
Hello, Using SA 3.2.5 I read in the Mail::SpamAssassin::Conf man page that: report_safe ( 0 | 1 | 2 ) (default: 1) ... If this option is set to 0, incoming spam is only modified by adding some "X-Spam-" headers and no changes will be made to the body. In addition, a header named X-Spam-Report will be added to spam. I am currently reconfiguring SA, and have set report_safe to 0. Our 'required' score is 8, and I have also configured: clear_report_template report "Score=_SCORE_ tests=_TESTS_ autolearn=_AUTOLEARN_" However, as far as I can tell, the X-Spam-Report header gets added to ham mail as well as spam. For example: X-spam-report: Score=-6.9 tests=BAYES_00,DCC_CHECK,RCVD_IN_DNSWL_HI autolearn=ham (taken from a received message; line wrapped be me). I have no problem with the header being added, and in fact that is what I wanted. However, I am a bit confused because the man page says it should only be added for spam mail. Can someone clarify what is going on please. Is there anything I need to do to the config to ensure that the above report is added to all mail (despite is seeming to happen anyway)? Thanks, John. -- ------- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: john.ho...@plymouth.ac.uk Fax: +44 (0)1752 587001
Spamd crash - redhat startup script problem?
Hello, Using: spamassassin 3.2.5 on a CentOS 5.2 system. Unfortunately the spamd process on one of our mail servers crashed early this morning. The system mail log showed: == Jan 31 06:52:00 tracy spamd[23255]: spamd: connection from localhost.localdomain [127.0.0.1] at port 45028 Jan 31 06:52:13 tracy spamd[2347]: spamd: server killed by SIGTERM, shutting down Jan 31 06:52:24 tracy spamd[26043]: server socket setup failed, retry 1: spamd: could not create INET socket on 127.0.0.1:783: Address already in use Jan 31 06:52:25 tracy spamd[23255]: spamd: checking message <200901310651.n0v6pxad026...@isg-prod-loader.informa.com> for sauser:10001 Jan 31 06:52:25 tracy spamd[26043]: server socket setup failed, retry 2: spamd: could not create INET socket on 127.0.0.1:783: Address already in use Jan 31 06:52:26 tracy spamd[26043]: spamd: could not create INET socket on 127.0.0.1:783: Address already in use Jan 31 06:52:31 tracy spamd[23255]: spamd: clean message (-6.6/8.0) for sauser:10001in 30.9 seconds, 5194 bytes. Jan 31 06:52:31 tracy spamd[23255]: spamd: result: . -6 - BAYES_00,RCVD_IN_DNSWL_MEDscantime=30.9,size=5194,user=sauser,uid=10001,required_score=8.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=45028,mid=<200901310651.n0v6pxad026...@isg-prod-loader.informa.com>,bayes=0.00,autolearn=ham Jan 31 06:52:31 tracy spamd[23255]: syswrite() to parent failed: Broken pipe at /usr/lib/perl5/vendor_perl/5.8.8/Mail/SpamAssassin/SpamdForkScaling.pm line 576. == My first thought was a bug in the SpamdForkScaling.pm module, but I'm not so sure. At 06:52 spamd was fine, but we have an sa-update/sa-compile job that runs at around that time. The files in /var/lib/spamassassin/compiled indicate that the job was running (or finishing) at 06:52. The job (if successful) then restarts spamassassin (using 'service spamassassin restart'). Now, the above log shows that at 06:52:13 SA received a shutdown signal - which is correct when restarting. But at 06:52:24 it seems to be trying to startup but cannot because SA is still running (the port is in use). Then at 06:52:31 it seems that some SA scan now finishes, and because SA was trying to restart, the parent process was gone and, hence, the syswrite error. Okay, so looking at the SA startup script it shows (this is within a shell 'case' statement): == stop) # Stop daemons. echo -n $"Stopping $prog: " killproc spamd RETVAL=$? echo if [ $RETVAL = 0 ]; then rm -f /var/lock/subsys/spamassassin rm -f $SPAMD_PID fi ;; restart) $0 stop sleep 3 $0 start ;; == I suspect the problem is that the 'stop' actually failed (RETVAL != 0). But since the 'restart' doesn't check this, it then just went on and tried to 'start' SA. This failed because SA still had a process/child running. Ultimately it meant that our mail server ended up with SA not running. Perhaps the RedHat (and hence Fedora (I assume)/CentOS) startup script should be a bit more aggressive in its checking that SA has actually stopped before trying to start it again? I think I would rather that more time was spent on ensuring that SA was stopped, so that it could then start, rather than it completely failing and the server being left without SA running. John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: john.ho...@plymouth.ac.uk Fax: +44 (0)1752 587001
Re: sought rules updates
On Tue, 2008-12-09 at 22:54 -0700, LuKreme wrote: > On 9-Dec-2008, at 17:09, John Horne wrote: > > Try: > > > >sa-update --gpgkey 6C6191E3 --channel sought.rules.yerp.org > > Ok, that gives me no error (where did you find/get the 6C6191E3?). It > sits for about 20-30 seconds and then I get a prompt back. But as far > as I can tell, nothing has changed. There is no new .cf file in /etc/ > mail/spamassassin (which is a link /etc/mail/spamassassin -> ../../usr/ > local/etc/mail/spamassassin if that matters), for example. > Look in '/var/lib/spamassassin/3*' within there there should be a new subdirectory and .cf file. John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 587001
Re: sought rules updates
On Tue, 2008-12-09 at 16:50 -0700, LuKreme wrote: > On 9-Dec-2008, at 12:58, Bill Landry wrote: > > Both the official SA rules and 3rd party rules can be updated via > > sa-update. For information and instructions, see: > > > > http://daryl.dostech.ca/sa-update/sare/sare-sa-update-howto.txt > > Ah yes, I remember a lot of those from the days run rjd. Geez there's > a lot of them... and they look like they are very old, with last > updated dates in 2005-2006 and none newer than Aug 2007. > > I tried this: > > $ cd /etc/mail/spamassassin > $ wget http://yerp.org/rules/GPG.KEY >% Total% Received % Xferd Average Speed TimeTime > Time Current > Dload Upload Total Spent > Left Speed > 100 2437 100 24370 0 10583 0 --:--:-- --:--:-- > --:--:-- 1291k > $ sa-update --import GPG.KEY > $ sa-update --channel sought.rules.yerp.org > error: GPG validation failed! > Try: sa-update --gpgkey 6C6191E3 --channel sought.rules.yerp.org John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 587001
Logging additional info on rule matches
Hello, Using SA 3.2.5 I was wondering it is possible to get SA to log additional information when a rule matches? For example, if I create a simple rule such as: body LOCAL_PWD_CHK/password/ to see if the word 'password' is in the message body, then I would probably want to have things such as the Subject line and the sender logged. Is this possible - perhaps by invoking 'logger'? Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 587287 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 587001
Re: pyzor: check failed: internal error
On Sat, 2006-09-09 at 12:58 -0500, John Thompson wrote: > > Ok, this suggests that the error producing the internal error messages > at that time was patched with pyzor-0.4.0. I'm running pyzor-0.4.0_4, > which presumably includes the needed patch. > Not necessarily. Pyzor 0.4.0 from original source does not include the mentioned patches (obviously). Likewise, running pyzor (pyzor-0.4.0-9.fc4) under FC4 does not include the patches (neither as far as I can see will the upcoming FC6). However, Debian pyzor seems to be patched. You'll need to check the FreeBSD source of your running version to see if it has the patches applied. Having said all that, under FC4 pyzor (patched) still seems to show these errors. John. -- ------- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: Spamd child states?
On Wed, 2006-09-06 at 17:17 +0100, John Horne wrote: > > > I get the feeling that something is wrong here. I have restarted SA, and > grepped the log file. It shows: > > === > prefork: child states: BI > prefork: child states: BB > prefork: child states: BBB > prefork: child states: > prefork: child states: S > prefork: child states: II > prefork: child states: IBBBII > prefork: child states: IIBBIK > prefork: child states: IIIBKK > prefork: child states: IIKIKK > prefork: child states: IB > prefork: child states: II > prefork: child states: BB > prefork: child states: BBB > [snipped] I investigated this further last night when our server was less busy. Below is the message I sent to Justin Mason explaining what I think is happening. The problem lies with SElinux. Under FC4 I cannot see anything I can turn on/off in selinux to resolve this, so we will need to run the server with selinux disabled. I suspect selinux needs a little tweak to allow both SA and selinux to run. > Hello, > > I noticed that always the first 2 child processes started remained > working okay. I assume that these 2 were related to the --min-children > and --min-spare options. All the children options, except > --max-children, are default in our configuration. However, any > subsequent child process started falls in to the 'K' state and seems > to remain there. > > Our servers are quieter at this time of night (midnight!), so I > straced the master process after killing all the children again. The > spamd maillog shows (using tail -f maillog|grep 'spawned child'): > > > Sep 7 00:20:42 tracy spamd[1666]: spamd: server successfully spawned > child process, pid 16267 > Sep 7 00:20:42 tracy spamd[1666]: spamd: server successfully spawned > child process, pid 16268 > Sep 7 00:21:36 tracy spamd[1666]: spamd: server successfully spawned > child process, pid 16341 > > > > The attached log shows, for pid 16341, that the kill call gives an > error - Operation not permitted. This explains why the child is not > killed, but not as to why the op is not permitted. > > The server is running Fedora Core 4 Linux, and has SElinux enabled. I > temporarily disabled selinux, and that seems to have resolved the > problem. An strace at the time (not attached) shows: > >[pid 1666] kill(19990, SIGINT) = 0 > > No error message. Also the maillog shows: > > === > Sep 7 00:46:07 tracy spamd[1666]: prefork: child states: BB > Sep 7 00:46:07 tracy spamd[1666]: prefork: child states: BBI > Sep 7 00:46:09 tracy spamd[1666]: prefork: child states: IBI > Sep 7 00:46:09 tracy spamd[1666]: prefork: child states: III > Sep 7 00:46:09 tracy spamd[1666]: prefork: child states: II > === > > As can be seen the new children process is successfully killed off. > > So I guess now I need to see what it actually is in selinux that is > stopping the master process from killing of its child processes. That > can wait till tomorrow. John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: Spamd child states?
On Wed, 2006-09-06 at 17:35 +0100, Justin Mason wrote: > > That looks bad :( The strace snippet, however, is pretty normal-looking. > > First off, are you using an up-to-date 3.1.x release? > Yes, version 3.1.4. > Secondly, you need to strace both the child *and* the parent spamd process > -- the easiest way to do this is to "strace -f" the parent spamd, then > kill -15 the kids so it starts new (traced) ones. > Okay, I did that. It ran for a few minutes and produced a 10MB file. What is odd is that while strace was running the log file shows for the child states: === prefork: child states: BB prefork: child states: BS prefork: child states: BBS prefork: child states: BBBS prefork: child states: S prefork: child states: BS prefork: child states: BBS prefork: child states: BBBS prefork: server reached --max-children setting, consider raising it prefork: child states: prefork: server reached --max-children setting, consider raising it prefork: child states: prefork: server reached --max-children setting, consider raising it prefork: child states: prefork: server reached --max-children setting, consider raising it prefork: child states: === This then goes on for quite a bit, but the child state remains at ''. Now that I have stopped strace, the state is: === prefork: child states: BIKK prefork: child states: IIKK prefork: child states: IIKK prefork: child states: IIKK prefork: child states: IBKK === John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: Spamd child states?
On Wed, 2006-09-06 at 11:38 -0400, Theo Van Dinter wrote:
> My understanding (I haven't really looked at that code) is that "K" means the
> child has been killed but it hasn't exited yet. If a child is in that state
> for more than, say, 5 seconds, there's likely an issue where it doesn't
> actually die off, imo.
>
> You should generally see states of I or B.
>
I get the feeling that something is wrong here. I have restarted SA, and
grepped the log file. It shows:
===
prefork: child states: BI
prefork: child states: BB
prefork: child states: BBB
prefork: child states:
prefork: child states: S
prefork: child states: II
prefork: child states: IBBBII
prefork: child states: IIBBIK
prefork: child states: IIIBKK
prefork: child states: IIKIKK
prefork: child states: IB
prefork: child states: II
prefork: child states: BB
prefork: child states: BBB
prefork: child states: BBBB
prefork: server reached --max-children setting, consider raising it
prefork: child states: BIBB
prefork: child states: IBBB
prefork: child states: IBIB
prefork: child states: IIIB
prefork: child states: BIKI
prefork: child states: IBKB
prefork: child states: BBKI
prefork: child states: BIKI
prefork: child states: IIKI
prefork: child states: IBKK
prefork: child states: IIKK
prefork: child states: BBKK
prefork: server reached --max-children setting, consider raising it
prefork: child states: BBKK
prefork: server reached --max-children setting, consider raising it
prefork: child states: IBKK
prefork: child states: BIKK
prefork: child states: IIKK
===
Some of the processes seem to almost immediately go in to the 'killed'
state and stay there. 'ps auxww' shows that all 8 child processes are
started. Running an strace (this is a Fedora Core 4 server) on some of
the processes seems to show that they are waiting on select, and then
get a 'resources unavailable' error. What resource I have no idea. E.g:
===
strace -Ff -p 12805
Process 12805 attached - interrupt to quit
select(16, [10], NULL, NULL, {290, 888000}) = 1 (in [10], left {147,
82})
read(10, "P\n", 6) = 6
read(10, 0xb4515f0, 6) = -1 EAGAIN (Resource
temporarily unavailable)
time(NULL) = 1157559274
select(16, [10], NULL, NULL, {300, 0}
===
The process just sits there in this loop of some sort, and never seems
to do any actual spam processing.
Any ideas about this?
Thanks,
John.
--
---
John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914
E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
RE: Spamd child states?
On Wed, 2006-09-06 at 16:03 +0100, John Horne wrote: > > > The server has 2GB of ram. It runs an MTA and SA, but does not do virus > checking. I only installed 3.1.4 yesterday, so it is a little early to > say if there are problems. However, I am seeing in the logs messages > like these: > >Sep 6 15:58:11 tracy spamd[13052]: prefork: server reached >--max-children setting, consider raising it > > SA seems to indicate that the value should be raised rather than > lowered. > Actually these messages seem to be a bit confusing. The log shows: Sep 6 16:05:39 tracy spamd[13052]: prefork: child states: KKBB Sep 6 16:05:39 tracy spamd[13052]: prefork: server reached --max-children setting, consider raising it Which seems to indicate that 2 children are busy but 6 have been killed. Why should I then raise the value? Surely it should just restart one of the killed children? If it said '' then I would understand it suggesting to raise the value. John. -- ------- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
RE: Spamd child states?
On Wed, 2006-09-06 at 09:35 -0400, Bowie Bailey wrote: > John Horne wrote: > > Hello, > > > > Running SA 3.1.4, I see messages such as these in the log file: > > > > Sep 6 00:05:21 tracy spamd[1710]: prefork: child states: KKI > > > > > From the code the various letters seem to indicate killed, > > > initialised, > > busy etc. My question though is are these just informational type > > messages? Are they something I need to take note of or monitor? > > Not unless they are causing a problem. These are just status messages > so you can see how the child processes are being used. > > One thing I note is that you have set your max-children to at least 7. > Yes, it is set to 8. The above message was taken soon after an SA restart, so perhaps only 7 had started at that time. > With this many children, keep a close eye on your memory usage. You > should generally allow about 50M of ram for each child. With 7 > children, you will need 350M of memory just for SA. With other stuff > running, this can easily be too much for a 1GB server. If you start > running into performance problems, you might want to take a closer > look at this. > The server has 2GB of ram. It runs an MTA and SA, but does not do virus checking. I only installed 3.1.4 yesterday, so it is a little early to say if there are problems. However, I am seeing in the logs messages like these: Sep 6 15:58:11 tracy spamd[13052]: prefork: server reached --max-children setting, consider raising it SA seems to indicate that the value should be raised rather than lowered. John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Spamd child states?
Hello, Running SA 3.1.4, I see messages such as these in the log file: Sep 6 00:05:21 tracy spamd[1710]: prefork: child states: KKI >From the code the various letters seem to indicate killed, initialised, busy etc. My question though is are these just informational type messages? Are they something I need to take note of or monitor? Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: Incorrect message-id logged?
On Tue, 2006-08-29 at 10:25 -0400, Theo Van Dinter wrote: > On Tue, Aug 29, 2006 at 03:06:54PM +0100, John Horne wrote: > > scantime=5.2,size=3097,mid=,bayes=1, > > > > I have searched through the MTA logs for the past month, and used a > > shortened part of the above mid, but nothing was found. So the question > > is how come SA shows a 'mid=dfee66ed26bce3c839092a95248645c8@' value but > > the MTA log files show nothing? > > > > Anyone have any ideas about this? > > MTA is broken? > Nah :-) > MTA generates the Message-ID (so it logs nothing coming in, > but then SA sees one)? > Nope, the MTA doesn't generate a Message-ID header. > Message actually has no Message-ID but has a > Resent-Message-ID? > > Without seeing the message it's hard to say. > Unfortunately I don't have all the headers for the message. I'll see if I can get the MTA to log when a message arrives with no Message-ID but does have a Resent-Message-ID. I can then compare what is logged by the MTA with the SA log. Not sure if the MTA would, by default, log the Resent-Message-ID if no Message-ID was present. I'll see if I can find out, and perhaps suggest it if it doesn't. Thanks for the replies, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Incorrect message-id logged?
Hello, We run site-wide SA (3.0.6), and I was asked to investigate a problem (what exactly is not relevant). I noticed in our SA log entries such as: === Aug 29 09:56:17 mary spamd[23037]: result: Y 13 - BAYES_99,FH_SALES_REPLY,HTML_50_60,HTML_FONT_BIG,HTML_IMAGE_ONLY_20, HTML_MESSAGE,INVALID_MSGID,MIME_HTML_ONLY,MSGID_NO_HOST,RAZOR2_CF_RANGE_51_100, RAZOR2_CHECK,RCVD_IN_SORBS_LOCAL,URIBL_BLACK_LOCAL scantime=5.2,size=3097,mid=,bayes=1, autolearn=no === Now I assume the 'mid=' part above refers to the Message-ID header? Our MTA, Exim, also logs the Message-ID in its own log files. I have, I think, located the arrival of the above message in the Exim logs (based on date/time, sender, recipient), but it indicates that there was no Message-ID header. I have searched through the MTA logs for the past month, and used a shortened part of the above mid, but nothing was found. So the question is how come SA shows a 'mid=dfee66ed26bce3c839092a95248645c8@' value but the MTA log files show nothing? Anyone have any ideas about this? Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: Whitelist_subject and Blacklist_Subject
On Wed, 2006-07-19 at 15:57 +0530, Ramprasad wrote: > On Mon, 2006-07-17 at 14:04 -0300, Claudia Burman wrote: > > I've googled and I searched the list archives but I can't find > > information on this. > > How do you use the whitelist subject and the blacklist subject plugin? > > Where do yo write the blacklist or the whitelist? > > > > Thanks > > Claudia Burman > > El Bolsón, Patagonia Argentina > > http://spamassassin.apache.org/full/3.1.x/dist/doc/Mail_SpamAssassin_Plugin_WhiteListSubject.html > > just put in your local.cf ( or wherever you want to ) > > > whitelist_subject good subject > blacklist_subject spammy subject > > Assuming you have the subject module loaded in SA > Hello, I'm guessing here that this is an SA 3.1 thing (subject whitelisting)? We are running 3.0.6. My question though is does whitelisting something cause SA to abort trying the other tests? In our case we tend to have senders, and sometimes subjects, that we want to allow through. To that extent we don't need/want SA to carry out any other tests, as by whitelisting them we are saying we don't care about the message just let it through. It seems a waste of the servers (and network) resources to carry out a load of tests when we know, by doing the whitelist tests first, that we want the message delivered unmarked. Regards, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Missing msgid check?
Hello,
I noticed in our log that some messages were being reported with no
Message-Id header ('mid=(unknown)'):
Jun 19 02:13:14 mary spamd[9149]: result: . 2 -
BAYES_00,HTML_MESSAGE,HTML_OBFUSCATE_10_20,J_CHICKENPOX_63,J_CHICKENPOX_73,J_CHICKENPOX_93,MIME_HTML_ONLY,RCVD_IN_BL_SPAMCOP_NET,SARE_UNI
scantime=1.6,size=29978,mid=(unknown),bayes=5.55111512312578e-17,autolearn=no
Checking to see if there was a rule to mark messages with no message-id
I came across this from a short while ago:
http://mail-archives.apache.org/mod_mbox/spamassassin-users/200603.mbox/[EMAIL
PROTECTED]
However, in there it says that the rule MSGID_FROM_MTA_ID would be
activiated. As can be seen from the log, it isn't. Our own MTA does not
'fix' messages without the message-id header, so the message must have
arrived without it.
Is this a bug with SA (the rule not being used when it should), or do I
need to create an actual rule to check for a missing Message-Id header?
We are using SA version 3.0.6.
Thanks,
John.
--
-------
John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914
E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: iXhash plugin docs updated, version for 3.0.x added.
On Wed, 2006-06-21 at 23:00 +0200, Chr. v. Stuckrad wrote: > On Wed, 21 Jun 2006, Dirk Bonengel wrote: > > > - added a version that runs under SpamAssassin 3.0.x > > Thanks a lot! After shortening some of the descriptions > (my --lint complains because of more than 50 chars) > it already caught some spams this evening! > Likewise, many thanks for this. I've kept the scores low for these tests for the moment just to see how many mails would be marked as spam. The log indicates that the tests would have flagged some spam mail already! John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: How does SA detect non-english language?
On Sat, 2005-08-27 at 10:19 -0700, Robert Menschel wrote: > > JH> X-Spam-Status: Yes, score=13.7 required=8.0 tests=BAYES_99,HTML_20_30, > JH> HTML_MESSAGE,MANGLED_LOOK,SARE_HTML_P_MANY3,SARE_RAND_2, > JH> SARE_RECV_IP_218216,SARE_SUB_ENC_ISO2022JP,SARE_SUB_PCT_LETTER, > JH> SUBJ_ALL_CAPS autolearn=unavailable version=3.0.4 > > JH> Unfortunately at the time I had left included in our site-wide > JH> configuration some of the specific 'ENG' SARE rules, so that explains > JH> the SARE_SUB_ENC_ISO2022JP matching and bumping the score up a bit. The > JH> SARE_RECV_IP_218216 is also a bit worrying (the message may have passed > JH> through a known spam relay). > > If you're using the latest SARE version, SARE_RECV_IP_218216 should be > scoring only 0.964, because we have detected ham coming through that > range of servers (though spam:ham > 100:1). If you can send me some > confirmed ham (full emails, headers and all), I can add those to my > corpus and that will help drive the score down. > [snipped] Hello, Many thanks, and to Matt Kettler, for your suggestions about this. I have now removed the specific ENG rules, and the mangled.cf. Unfortunately I cannot send you a copy of the message itself since I do not have it. I have asked the student for a copy of the full message, but so far have received nothing. Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
How does SA detect non-english language?
Hello, We have had a complaint from a user that some of his Japanese mail (being received by us) is always marked by SA as spam. As a University it is natural for us to receive foreign mail messages. However, what I am unsure about is how does SA detect a foreign (non-English) language? The user has only sent me the message headers so far, and the relevant bits show: === From: =?ISO-2022-JP?B?GyRCGyRCIXkbKEI=?= maki =?ISO-2022-JP?B?GyRAGyRCIXkbKEI=?= <[EMAIL PROTECTED]> Subject: ***SPAM*** (13.7) (=?ISO-2022-JP?B?GyRCJU4bKEI=?= _ =?ISO-2022-JP?B?GyRCISUhIxsoQg==?= ) X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on tracy.csd.plymouth.ac.uk X-Spam-Level: * X-Spam-Status: Yes, score=13.7 required=8.0 tests=BAYES_99,HTML_20_30, HTML_MESSAGE,MANGLED_LOOK,SARE_HTML_P_MANY3,SARE_RAND_2, SARE_RECV_IP_218216,SARE_SUB_ENC_ISO2022JP,SARE_SUB_PCT_LETTER, SUBJ_ALL_CAPS autolearn=unavailable version=3.0.4 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="--=_43041108.8435A812" === As can be seen the From: and Subject: headers are encoded. If the body was in Japanese too then wouldn't the MIME headers have some indication of the encoding too? Unfortunately at the time I had left included in our site-wide configuration some of the specific 'ENG' SARE rules, so that explains the SARE_SUB_ENC_ISO2022JP matching and bumping the score up a bit. The SARE_RECV_IP_218216 is also a bit worrying (the message may have passed through a known spam relay). As it is SA sees the body as HTML and I am wondering if the mixture of HTML and Japanese in the body is causing the message to be scored high - I am assuming here that SA doesn't realise that the body is Japanese and so treats it as nonsensical English? Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
RE: RDJ from cron - is it safe?
On Fri, 2005-06-24 at 15:53 -0400, Chris Santerre wrote: > > I'm completely guessing out of the blue here, but is it a timing issue? Is > it trying to restart before the final child is able to quit? > Yes, sort of... Chris Thielen wrote: > > Maybe try changing your SA_RESTART to "killall -HUP spamd". I think > spamd will correctly reload configuration files with a HUP signal > Oddly enough for some reason when I did a 'killall -HUP spamd' all the spamd processes were killed off rather than restarted! Secondly, the spamd man page mentions a warning about using HUP. It turns out that the problem is not with SA, or RDJ, as such, but with Fedora. To 'restart' SA, the fedora startup script issues a 'stop' then a 'start'. To help with this the 'stop' uses some script functions in /etc/init.d/functions. In particular it uses 'killproc'. That function tries to locate the pids of the spamd processes. It first looks for a pid file, /var/run/spamd.pid, but that doesn't exist. So it then calls the 'pidof' command. This returns a list of the pids. However, it seems that the child processes are listed first, so the last pid is the parent one. It seems that as the children are being killed off, the parent sees this and restarts a child! Hence the overall 'restart' fails. I have inserted 2 lines into 'killproc' to reverse the pid order. Now the parent pid is seen first and killed off first. Testing this, 'restart' now works fine every time. I'll submit this as a bug to Fedora Core to see what they say. Needless to say, this problem may be particular to Fedora, other unix/linuxes may handle restarts and/or pids differently, and so not see this problem. Many thanks for all the replies. Now we have sorted this 'bug' out, I'll see about automating RDJ :-) John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
Re: RDJ from cron - is it safe?
On Thu, 2005-06-23 at 14:13 -0700, Ed Kasky wrote: > At 09:54 AM Thursday, 6/23/2005, John Horne wrote -=> > >Hello, > > > >We have been running RDJ manually, but are now considering running it > >via cron. The problem is what if something 'goes wrong'? This is on a > >central mailhub, and we do not want the mail going through un-spam > >checked. I gather others do run RDJ from cron, so the question is > >have there been problems doing this? > > RDJ will not restart the daemon or even keep changed rulesets if the > lint returns an error. So, running it via a cron job is safe. > Many thanks for all the replies, which all seem positive. However, we have been seeing problems with restarting the daemon recently, which is why I am wary about starting to run RDJ from cron. In trying to restart spamassassin, on a fedora core 4 and core 3 system, we see: /etc/init.d/spamassassin restart Shutting down spamd: [ OK ] Starting spamd: Could not create INET socket on 127.0.0.1:783: Address already in use (IO::Socket::INET: Address already in use) [FAILED] It seems that a single child procees is left running: ps auxww|grep -i spamd mail 4156 0.0 2.7 61532 57152 ?S17:28 0:00 spamd child root 4169 0.0 0.0 3756 736 pts/1S+ 17:28 0:00 grep -i spamd If we run 'restart' again then it works okay. If we do a stop and then a start, that too works okay. Does anyone else see this problem? Our mail servers can get busy, so we start SA with the options: -d -x -m 15 -s daemon -u mail --max-conn-per-child=100 Reducing the '-m' value made no difference to this problem. SA version is 3.0.4 on the FC4 server. Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
RDJ from cron - is it safe?
Hello, We have been running RDJ manually, but are now considering running it via cron. The problem is what if something 'goes wrong'? This is on a central mailhub, and we do not want the mail going through un-spam checked. I gather others do run RDJ from cron, so the question is have there been problems doing this? Thanks, John. -- --- John Horne, University of Plymouth, UK Tel: +44 (0)1752 233914 E-mail: [EMAIL PROTECTED] Fax: +44 (0)1752 233839
