Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
From: Phil Barnett [EMAIL PROTECTED]
On Friday 22 June 2007 00:54, jdow wrote:
I think it was mentioned around these precincts about the time tripwire
was converted to 99_FVGTTripWire.cf and added to the SARE repositories
as a SARE rule set. I also note that I don't use it here anymore. The
return on CPU cycles investment was not sufficient to run that set
anymore.
When I'm looking for a place to shed load, I'll remember. Right now, this
is a
quad processor box, so I'll take all the rules I can get. We have a pretty
good spam marking rate right now. Not many things hit tripwire, but all
the
ones that do are spam, so I find it useful to drive the score up.
Take a quick look at tripwire and its newer equivalent. They should be
about the same thing. Loading both will result in the rules that may share
a name between the files having the newer version superseded by the older
version because files load in alphabetical order.
{^_^}
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Friday 22 June 2007 12:32, jdow wrote: Take a quick look at tripwire and its newer equivalent. They should be about the same thing. Loading both will result in the rules that may share a name between the files having the newer version superseded by the older version because files load in alphabetical order. I checked. RDJ is pulling the new one and naming it tripwire.cf in the working rule directory. At least they have the same date/time stamp and identical content. So I think I'm only using the newer one. Thanks. -- Phil Barnett AI4OF SKCC #600
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
From: Phil Barnett [EMAIL PROTECTED]
On Friday 22 June 2007 12:32, jdow wrote:
Take a quick look at tripwire and its newer equivalent. They should be
about the same thing. Loading both will result in the rules that may
share
a name between the files having the newer version superseded by the older
version because files load in alphabetical order.
I checked. RDJ is pulling the new one and naming it tripwire.cf in the
working
rule directory. At least they have the same date/time stamp and identical
content. So I think I'm only using the newer one.
RDJ does THAT? That's unbelievably ugly! The SARE rules have the lead
numbers for a purpose, make sure the rules load in a specific order.
{O.O} Me glad me use me own bash script instead of RDJ me thinks.
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Hope that helps Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Nigel Frankcom wrote: I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Note that there haven't been any updates to 70_sare_stocks.cf since May 7th and no updates at all since June 5th, so manual updates probably aren't worth the bother. Daryl [EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Nigel Frankcom wrote: On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Matt
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thu, 21 Jun 2007 03:30:00 -0400, Daryl C. W. O'Shea [EMAIL PROTECTED] wrote: Nigel Frankcom wrote: I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Note that there haven't been any updates to 70_sare_stocks.cf since May 7th and no updates at all since June 5th, so manual updates probably aren't worth the bother. Daryl [EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$ It's good to know there's been no updates; though I'd guessed that from the file time stamps on rulesemporium. There still seems to be a problem with RDJ though. It looks like it's pulling an entire page not just rules; I can't see any other reason for the table etc elements in the debug. I'm still curious as to why so many stock spam are getting through (so many being relative to normal). On the surface they don't look any different from those that have been caught for ages. Samples available if required. Kind regards Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thu, 21 Jun 2007 09:38:03 +0200, Matthias Keller [EMAIL PROTECTED] wrote: Nigel Frankcom wrote: On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED] wrote: Is anyone else getting these failed messages on their tripwire.cf updates? I've been getting this message for several days now. It looks to me like the new tripwire.cf is very broken. -- Forwarded Message -- Subject: RulesDuJour Run Summary on taz5.fiberhosting.net Date: Thursday 21 June 2007 02:26 From: To: RulesDuJour Run Summary on taz5.fiberhosting.net: TripWire has changed on taz5.fiberhosting.net. Version line: ***WARNING***: spamassassin --lint failed. Rolling configuration files back, not restarting SpamAssassin. Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf /usr/share/spamassassin/RulesDuJour/99_ --- FVGT_Tripwire.cf.2; mv -f /usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225 /usr/share/spamassassin/tripwire.cf; Lint output: [24363] warn: config: failed to parse line, skipping: HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Pragma CONTENT=no-cache [24363] warn: config: failed to parse line, skipping: META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues detected, please rerun with debug enabled for more information I've been getting the same for weeks. I ended up manually updating rules; especially the stock one since more and more seem to be slipping through. The problems seemed to start after the DDoS on rulesemporium; since then I've not been able to get any sense out of it via RDJ. When I manually update it all lint's clean. Time consuming but it works Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Matt Give that man a cigar! Seemed to work OK. Thanks Matt. Kind regards Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thursday 21 June 2007 03:38, Matthias Keller wrote: Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Did that two days ago. And everything came in fine and worked. I linted it then and tonight and the current ruleset lints fine. The error messages are from the RDJ script pulling in a new file. It does look like the RDJ script is pulling the wrong file because the lint error shows html tags and there aren't any in my current tripwire.cf file. If it is true that there are no updates, then why is the RDJ script trying to update anything? Is the RDJ server still being DOS'd? -- Phil Barnett AI4OF SKCC #600
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Phil Barnett wrote: On Thursday 21 June 2007 03:38, Matthias Keller wrote: Just try to delete the downloaded files in your rules_du_jour folder (for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just the rule(s) that go wrong.I then redownloads the rules correctly and you're clear to go with RDJ again Did that two days ago. And everything came in fine and worked. I linted it then and tonight and the current ruleset lints fine. The error messages are from the RDJ script pulling in a new file. It does look like the RDJ script is pulling the wrong file because the lint error shows html tags and there aren't any in my current tripwire.cf file. If it is true that there are no updates, then why is the RDJ script trying to update anything? Is the RDJ server still being DOS'd The Problem occurs because RDJ once fetched a page which wasn't the rules but a HTML page. Now the config didn't lint anymore but RDJ doesn't delete the offending page but assumes, that the page will be updated soon to fix the lint error. The downloaded HTML page now has a current timestamp and since the real .cf file is older and there hasn't been an update since, the newer HTML page is kept and fails to lint every time until you manually delete it (and the actual .cf gets redownloaded) or a new version of the .cf is uploaded... RDJ ONLY redownloads a rule from the server IF it has been modified since the timestamp of the local file... If RDJ would delete failing files, it would solve that problem but lead to more traffic to the server, especially when ddoses happen, because everyone would try to redownload all the pages all the times Matt
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Phil Barnett schrieb:
On Thursday 21 June 2007 03:38, Matthias Keller wrote:
Just try to delete the downloaded files in your rules_du_jour folder
(for example /etc/mail/spamassassin/rules_du_jour/* ), respectively just
the rule(s) that go wrong.I then redownloads the rules correctly and
you're clear to go with RDJ again
Did that two days ago. And everything came in fine and worked. I linted it
then and tonight and the current ruleset lints fine.
The error messages are from the RDJ script pulling in a new file. It does look
like the RDJ script is pulling the wrong file because the lint error shows
html tags and there aren't any in my current tripwire.cf file.
If it is true that there are no updates, then why is the RDJ script trying to
update anything? Is the RDJ server still being DOS'd?
This (see post new patch for rules_du_jour ... (Lindsay
Haisley)/18.06.2007) works fine here.
But you probably have to delete the faulty .cf files manually.
cut here
--- /root/rules_du_jour.orig2007-06-17 21:01:24.0 -0500
+++ /var/lib/spamassassin/rules_du_jour 2007-06-18 12:37:44.0 -0500
@@ -907,6 +907,8 @@
[ ${SEND_THE_EMAIL} ] echo -e ${MESSAGES} | sh -c ${MAILCMD} -s
\RulesDuJour Run Summary on ${HOSTNAME}\ ${MAIL_ADDRESS};
fi
+grep -il 'META HTTP-EQUIV' ${TMPDIR}/*|xargs -n1 rm -f
+
cd ${OLDDIR};
exit;
cut here
--
Grüsse/Greetings
MH
Dont send mail to: [EMAIL PROTECTED]
--
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Unless something has changed with the most recent versions of SpamAssassin
I see two configuraton errors present.
1) YOu do NOT use /use/share/spamassassin to store rules. They belong
in /etc/mail/spamassassin or some other such place.
2) Why are you running tripwire.cf (obsolete) and 99_FGTTripWire.cf at
the same time?
(Note that I do NOT use RulesDuJour here. I have my own script for
handling updates. It's a ridiculously simple bash script that I can
read. {^_-} So maybe using /usr/share/spamassassin is some form of
silly RDJ action. That is a directory that is utterly wiped out with
each upgrade of SpamAssassin, though. So anything in it gets lost.)
{^_^}
- Original Message -
From: Nigel Frankcom [EMAIL PROTECTED]
On Thu, 21 Jun 2007 03:07:52 -0400, Phil Barnett [EMAIL PROTECTED]
wrote:
Is anyone else getting these failed messages on their tripwire.cf updates?
I've been getting this message for several days now.
It looks to me like the new tripwire.cf is very broken.
-- Forwarded Message --
Subject: RulesDuJour Run Summary on taz5.fiberhosting.net
Date: Thursday 21 June 2007 02:26
From:
To:
RulesDuJour Run Summary on taz5.fiberhosting.net:
TripWire has changed on taz5.fiberhosting.net.
Version line:
***WARNING***: spamassassin --lint failed.
Rolling configuration files back, not restarting SpamAssassin.
Rollback command is: mv -f /usr/share/spamassassin/tripwire.cf
/usr/share/spamassassin/RulesDuJour/99_
---
FVGT_Tripwire.cf.2; mv -f
/usr/share/spamassassin/RulesDuJour/tripwire.cf.20070621-0225
/usr/share/spamassassin/tripwire.cf;
Lint output: [24363] warn: config: failed to parse line, skipping:
HTMLHEADMETA HTTP-EQUIV=Refresh CONTENT=0.1 [24363] warn:
config:
failed to parse line, skipping: META HTTP-EQUIV=Pragma
CONTENT=no-cache [24363] warn: config: failed to parse line, skipping:
META HTTP-EQUIV=Expires CONTENT=-1 [24363] warn: config: failed to
parse line, skipping: /HEAD/HTML [24363] warn: lint: 4 issues
detected,
please rerun with debug enabled for more information
I've been getting the same for weeks. I ended up manually updating
rules; especially the stock one since more and more seem to be
slipping through.
The problems seemed to start after the DDoS on rulesemporium; since
then I've not been able to get any sense out of it via RDJ.
When I manually update it all lint's clean. Time consuming but it
works
Hope that helps
Nigel
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Daryl, note that a simple update to RDJ to add a time gap between
individual file update attempts gets through the DDoS protection
somewhat better than a raw RDJ. A friend of mine made such a change
and has it working better.
{^_^}
- Original Message -
From: Daryl C. W. O'Shea [EMAIL PROTECTED]
Nigel Frankcom wrote:
I've been getting the same for weeks. I ended up manually updating
rules; especially the stock one since more and more seem to be
slipping through.
The problems seemed to start after the DDoS on rulesemporium; since
then I've not been able to get any sense out of it via RDJ.
When I manually update it all lint's clean. Time consuming but it
works
Note that there haven't been any updates to 70_sare_stocks.cf since May
7th and no updates at all since June 5th, so manual updates probably
aren't worth the bother.
Daryl
[EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun
drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf
drwxrwxr-x 2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf
drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu0.cf
drwxrwxr-x 2 dos dos 4096 Jun 4 21:14 70_sare_obfu1.cf
drwxrwxr-x 2 dos dos 4096 May 7 00:24 70_sare_stocks.cf
drwxrwxr-x 2 dos dos 12288 May 24 12:14 70_sc_top200.cf
drwxrwxr-x 2 dos dos 4096 May 21 10:14 72_sare_bml_post25x.cf
[EMAIL PROTECTED] channels]$
RE: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
-Original Message-
From: jdow [mailto:[EMAIL PROTECTED]
Sent: Thursday, June 21, 2007 7:50 AM
To: users@spamassassin.apache.org
Subject: Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
Daryl, note that a simple update to RDJ to add a time gap
between individual file update attempts gets through the DDoS
protection somewhat better than a raw RDJ. A friend of mine
made such a change and has it working better.
I have been getting these same messages as Daryl since the DDoS attack
on rulesemporium also. I am not so good with all of this though. Could
you please explain in more detail what you mean by this statement? What
do you mean by adding a time gap? Perhaps I am asking an obvious
question but I am afraid your statement is not obvious to me.
Thanks,
Steve
{^_^}
- Original Message -
From: Daryl C. W. O'Shea [EMAIL PROTECTED]
Nigel Frankcom wrote:
I've been getting the same for weeks. I ended up manually updating
rules; especially the stock one since more and more seem to be
slipping through.
The problems seemed to start after the DDoS on
rulesemporium; since
then I've not been able to get any sense out of it via RDJ.
When I manually update it all lint's clean. Time consuming but it
works
Note that there haven't been any updates to 70_sare_stocks.cf since
May
7th and no updates at all since June 5th, so manual updates
probably
aren't worth the bother.
Daryl
[EMAIL PROTECTED] channels]$ ls -l | grep -P May|Jun
drwxrwxr-x 2 dos dos 4096 May 21 10:14 70_sare_adult.cf
drwxrwxr-x
2 dos dos 4096 Jun 5 11:14 70_sare_obfu.cf drwxrwxr-x 2 dos dos
4096 Jun 4 21:14 70_sare_obfu0.cf drwxrwxr-x 2 dos dos
4096 Jun 4
21:14 70_sare_obfu1.cf drwxrwxr-x 2 dos dos 4096 May 7 00:24
70_sare_stocks.cf drwxrwxr-x 2 dos dos 12288 May 24 12:14
70_sc_top200.cf drwxrwxr-x 2 dos dos 4096 May 21 10:14
72_sare_bml_post25x.cf [EMAIL PROTECTED] channels]$
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Thursday 21 June 2007 08:47, jdow wrote: Unless something has changed with the most recent versions of SpamAssassin I see two configuraton errors present. 1) YOu do NOT use /use/share/spamassassin to store rules. They belong in /etc/mail/spamassassin or some other such place. This is a configurable option in SA, so you can put it anywhere you want. Why the people at Plesk decided to put it there is beyond me, but it works, so I'm leaving it alone. Also, /usr/share/spamassassin may be wiped out on each upgrade, but /usr/share/spamassassin/rulesdujour is not, and the next run of RDJ repopulates the former. 2) Why are you running tripwire.cf (obsolete) and 99_FGTTripWire.cf at the same time? When RDJ downloads 99FGTTripWire.cf, it renames it to tripwire.cf when it moves it. I had no idea that tripwire was obsolete. Where is this information distributed? -- Phil Barnett AI4OF SKCC #600
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
From: Phil Barnett [EMAIL PROTECTED]
On Thursday 21 June 2007 08:47, jdow wrote:
Unless something has changed with the most recent versions of SpamAssassin
I see two configuraton errors present.
1) YOu do NOT use /use/share/spamassassin to store rules. They belong
in /etc/mail/spamassassin or some other such place.
This is a configurable option in SA, so you can put it anywhere you want.
Why
the people at Plesk decided to put it there is beyond me, but it works, so
I'm leaving it alone.
Also, /usr/share/spamassassin may be wiped out on each upgrade,
but /usr/share/spamassassin/rulesdujour is not, and the next run of RDJ
repopulates the former.
2) Why are you running tripwire.cf (obsolete) and 99_FGTTripWire.cf at
the same time?
When RDJ downloads 99FGTTripWire.cf, it renames it to tripwire.cf when it
moves it.
I had no idea that tripwire was obsolete. Where is this information
distributed?
I think it was mentioned around these precincts about the time tripwire
was converted to 99_FVGTTripWire.cf and added to the SARE repositories
as a SARE rule set. I also note that I don't use it here anymore. The
return on CPU cycles investment was not sufficient to run that set
anymore.
{^_^}
Re: Fwd: RulesDuJour Run Summary on taz5.fiberhosting.net
On Friday 22 June 2007 00:54, jdow wrote: I think it was mentioned around these precincts about the time tripwire was converted to 99_FVGTTripWire.cf and added to the SARE repositories as a SARE rule set. I also note that I don't use it here anymore. The return on CPU cycles investment was not sufficient to run that set anymore. When I'm looking for a place to shed load, I'll remember. Right now, this is a quad processor box, so I'll take all the rules I can get. We have a pretty good spam marking rate right now. Not many things hit tripwire, but all the ones that do are spam, so I find it useful to drive the score up. -- Phil Barnett AI4OF SKCC #600
