Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
From: Phil Barnett [EMAIL PROTECTED] On Friday 22 June 2007 00:54, jdow wrote: I think it was mentioned around these precincts about the time tripwire was converted to 99_FVGTTripWire.cf and added to the SARE repositories as a SARE rule set. I also note that I don't use it here anymore. The return on CPU cycles investment was not sufficient to run that set anymore. When I'm looking for a place to shed load, I'll remember. Right now, this is a quad processor box, so I'll take all the rules I can get. We have a pretty good spam marking rate right now. Not many things hit tripwire, but all the ones that do are spam, so I find it useful to drive the score up. Take a quick look at tripwire and its newer equivalent. They should be about the same thing. Loading both will result in the rules that may share a name between the files having the newer version superseded by the older version because files load in alphabetical order. {^_^}
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Friday 22 June 2007 12:32, jdow wrote: Take a quick look at tripwire and its newer equivalent. They should be about the same thing. Loading both will result in the rules that may share a name between the files having the newer version superseded by the older version because files load in alphabetical order. I checked. RDJ is pulling the new one and naming it tripwire.cf in the working rule directory. At least they have the same date/time stamp and identical content. So I think I'm only using the newer one. Thanks. -- Phil Barnett AI4OF SKCC #600
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
From: Phil Barnett [EMAIL PROTECTED] On Friday 22 June 2007 12:32, jdow wrote: Take a quick look at tripwire and its newer equivalent. They should be about the same thing. Loading both will result in the rules that may share a name between the files having the newer version superseded by the older version because files load in alphabetical order. I checked. RDJ is pulling the new one and naming it tripwire.cf in the working rule directory. At least they have the same date/time stamp and identical content. So I think I'm only using the newer one. RDJ does THAT? That's unbelievably ugly! The SARE rules have the lead numbers for a purpose, make sure the rules load in a specific order. {O.O} Me glad me use me own bash script instead of RDJ me thinks.
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Hope that helps Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Nigel Frankcom wrote: I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Note that there haven't been any updates to 70_sare_stocks.cf since May 7th and no updates at all since June 5th, so manual updates probably aren't worth the bother. Daryl [EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Nigel Frankcom wrote: On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Matt
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thu, 21 Jun 2007 03:30:00 -0400, Daryl C. W. O'Shea [EMAIL PROTECTED] wrote: Nigel Frankcom wrote: I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Note that there haven't been any updates to 70_sare_stocks.cf since May 7th and no updates at all since June 5th, so manual updates probably aren't worth the bother. Daryl [EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$ It's good to know there's been no updates; though I'd guessed that from the file time stamps on rulesemporium. There still seems to be a problem with RDJ though. It looks like it's pulling an entire page not just rules; I can't see any other reason for the table etc elements in the debug. I'm still curious as to why so many stock spam are getting through (so many being relative to normal). On the surface they don't look any different from those that have been caught for ages. Samples available if required. Kind regards Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thu, 21 Jun 2007 09:38:03 +0200, Matthias Keller [EMAIL PROTECTED] wrote: Nigel Frankcom wrote: On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Matt Give that man a cigar! Seemed to work OK. Thanks Matt. Kind regards Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thursday 21 June 2007 03:38, Matthias Keller wrote: Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Did that two days ago. And everything came in fine and worked. I linted it then and tonight and the current ruleset lints fine. The error messages are from the RDJ script pulling in a new file. It does look like the RDJ script is pulling the wrong file because the lint error shows html tags and there aren't any in my current tripwire.cf file. If it is true that there are no updates, then why is the RDJ script trying to update anything? Is the RDJ server still being DOS'd? -- Phil Barnett AI4OF SKCC #600
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Phil Barnett wrote: On Thursday 21 June 2007 03:38, Matthias Keller wrote: Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Did that two days ago. And everything came in fine and worked. I linted it then and tonight and the current ruleset lints fine. The error messages are from the RDJ script pulling in a new file. It does look like the RDJ script is pulling the wrong file because the lint error shows html tags and there aren't any in my current tripwire.cf file. If it is true that there are no updates, then why is the RDJ script trying to update anything? Is the RDJ server still being DOS'd The Problem occurs because RDJ once fetched a page which wasn't the rules but a HTML page. Now the config didn't lint anymore but RDJ doesn't delete the offending page but assumes, that the page will be updated soon to fix the lint error. The downloaded HTML page now has a current timestamp and since the real .cf file is older and there hasn't been an update since, the newer HTML page is kept and fails to lint every time until you manually delete it (and the actual .cf gets redownloaded) or a new version of the .cf is uploaded... RDJ ONLY redownloads a rule from the server IF it has been modified since the timestamp of the local file... If RDJ would delete failing files, it would solve that problem but lead to more traffic to the server, especially when ddoses happen, because everyone would try to redownload all the pages all the times Matt
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Phil Barnett schrieb: On Thursday 21 June 2007 03:38, Matthias Keller wrote: Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Did that two days ago. And everything came in fine and worked. I linted it then and tonight and the current ruleset lints fine. The error messages are from the RDJ script pulling in a new file. It does look like the RDJ script is pulling the wrong file because the lint error shows html tags and there aren't any in my current tripwire.cf file. If it is true that there are no updates, then why is the RDJ script trying to update anything? Is the RDJ server still being DOS'd? This (see post new patch for rules_du_jour ... (Lindsay Haisley)/18.06.2007) works fine here. But you probably have to delete the faulty .cf files manually. cut here --- /root/rules_du_jour.orig2007-06-17 21:01:24.0 -0500 +++ /var/lib/spamassassin/rules_du_jour 2007-06-18 12:37:44.0 -0500 @@ -907,6 +907,8 @@ [ ${SEND_THE_EMAIL} ] echo -e ${MESSAGES} | sh -c ${MAILCMD} -s \RulesDuJour Run Summary on ${HOSTNAME}\ ${MAIL_ADDRESS}; fi +grep -il 'META HTTP-EQUIV' ${TMPDIR}/*|xargs -n1 rm -f + cd ${OLDDIR}; exit; cut here -- Grüsse/Greetings MH Dont send mail to: [EMAIL PROTECTED] --
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Unless something has changed with the most recent versions of SpamAssassin I see two configuraton errors present. 1) YOu do NOT use /use/share/spamassassin to store rules. They belong in /etc/mail/spamassassin or some other such place. 2) Why are you running tripwire.cf (obsolete) and 99_FGTTripWire.cf at the same time? (Note that I do NOT use RulesDuJour here. I have my own script for handling updates. It's a ridiculously simple bash script that I can read. {^_-} So maybe using /usr/share/spamassassin is some form of silly RDJ action. That is a directory that is utterly wiped out with each upgrade of SpamAssassin, though. So anything in it gets lost.) {^_^} - Original Message - From: Nigel Frankcom [EMAIL PROTECTED] On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Hope that helps Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Daryl, note that a simple update to RDJ to add a time gap between individual file update attempts gets through the DDoS protection somewhat better than a raw RDJ. A friend of mine made such a change and has it working better. {^_^} - Original Message - From: Daryl C. W. O'Shea [EMAIL PROTECTED] Nigel Frankcom wrote: I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Note that there haven't been any updates to 70_sare_stocks.cf since May 7th and no updates at all since June 5th, so manual updates probably aren't worth the bother. Daryl [EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$
RE: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
-Original Message- From: jdow [mailto:[EMAIL PROTECTED] Sent: Thursday, June 21, 2007 7:50 AM To: users@spamassassin.apache.org Subject: Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net Daryl, note that a simple update to RDJ to add a time gap between individual file update attempts gets through the DDoS protection somewhat better than a raw RDJ. A friend of mine made such a change and has it working better. I have been getting these same messages as Daryl since the DDoS attack on rulesemporium also. I am not so good with all of this though. Could you please explain in more detail what you mean by this statement? What do you mean by adding a time gap? Perhaps I am asking an obvious question but I am afraid your statement is not obvious to me. Thanks, Steve {^_^} - Original Message - From: Daryl C. W. O'Shea [EMAIL PROTECTED] Nigel Frankcom wrote: I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Note that there haven't been any updates to 70_sare_stocks.cf since May 7th and no updates at all since June 5th, so manual updates probably aren't worth the bother. Daryl [EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thursday 21 June 2007 08:47, jdow wrote: Unless something has changed with the most recent versions of SpamAssassin I see two configuraton errors present. 1) YOu do NOT use /use/share/spamassassin to store rules. They belong in /etc/mail/spamassassin or some other such place. This is a configurable option in SA, so you can put it anywhere you want. Why the people at Plesk decided to put it there is beyond me, but it works, so I'm leaving it alone. Also, /usr/share/spamassassin may be wiped out on each upgrade, but /usr/share/spamassassin/rulesdujour is not, and the next run of RDJ repopulates the former. 2) Why are you running tripwire.cf (obsolete) and 99_FGTTripWire.cf at the same time? When RDJ downloads 99FGTTripWire.cf, it renames it to tripwire.cf when it moves it. I had no idea that tripwire was obsolete. Where is this information distributed? -- Phil Barnett AI4OF SKCC #600
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
From: Phil Barnett [EMAIL PROTECTED] On Thursday 21 June 2007 08:47, jdow wrote: Unless something has changed with the most recent versions of SpamAssassin I see two configuraton errors present. 1) YOu do NOT use /use/share/spamassassin to store rules. They belong in /etc/mail/spamassassin or some other such place. This is a configurable option in SA, so you can put it anywhere you want. Why the people at Plesk decided to put it there is beyond me, but it works, so I'm leaving it alone. Also, /usr/share/spamassassin may be wiped out on each upgrade, but /usr/share/spamassassin/rulesdujour is not, and the next run of RDJ repopulates the former. 2) Why are you running tripwire.cf (obsolete) and 99_FGTTripWire.cf at the same time? When RDJ downloads 99FGTTripWire.cf, it renames it to tripwire.cf when it moves it. I had no idea that tripwire was obsolete. Where is this information distributed? I think it was mentioned around these precincts about the time tripwire was converted to 99_FVGTTripWire.cf and added to the SARE repositories as a SARE rule set. I also note that I don't use it here anymore. The return on CPU cycles investment was not sufficient to run that set anymore. {^_^}
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Friday 22 June 2007 00:54, jdow wrote: I think it was mentioned around these precincts about the time tripwire was converted to 99_FVGTTripWire.cf and added to the SARE repositories as a SARE rule set. I also note that I don't use it here anymore. The return on CPU cycles investment was not sufficient to run that set anymore. When I'm looking for a place to shed load, I'll remember. Right now, this is a quad processor box, so I'll take all the rules I can get. We have a pretty good spam marking rate right now. Not many things hit tripwire, but all the ones that do are spam, so I find it useful to drive the score up. -- Phil Barnett AI4OF SKCC #600