Cron ~/svn/trunk/build/mkupdates/run_nightly | /usr/bin/tee /var/www/automc.spamassassin.org/mkupdates/mkupdates.txt

2017-11-27 Thread Cron Daemon
+ promote_active_rules
+ pwd
/usr/local/spamassassin/automc/svn/trunk
+ /usr/bin/perl build/mkupdates/listpromotable
HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
day 2 contains a --net mass-check! offsetting by an extra day
Use of "goto" to jump into a construct is deprecated at 
build/mkupdates/listpromotable line 85.
HTTP get: http://ruleqa.spamassassin.org/3-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/4-days-ago?xml=1
+ mv rules/active.list.new rules/active.list
+ svn diff rules
+ cat /var/www/ruleqa.spamassassin.org/reports/LATEST
Index: rules/active.list
===
--- rules/active.list   (revision 1816371)
+++ rules/active.list   (working copy)
@@ -74,6 +74,9 @@
 ALL_TRUSTED
 
 # good enough
+ANY_PILL_PRICE
+
+# good enough
 APOSTROPHE_TOCC
 
 # good enough
@@ -334,9 +337,6 @@
 # tflags publish
 FUZZY_UNSUBSCRIBE
 
-# good enough
-GAPPY_GENITALIA
-
 # tflags publish
 GOOGLE_DOCS_PHISH
 
@@ -401,6 +401,9 @@
 HK_NAME_DRUGS
 
 # good enough
+HK_RANDOM_ENVFROM
+
+# good enough
 HK_RANDOM_FROM
 
 # good enough
@@ -439,9 +442,6 @@
 # good enough
 MILLION_USD
 
-# good enough
-MIMEOLE_DIRECT_TO_MX
-
 # tflags userconf
 MIME_CHARSET_FARAWAY
 
@@ -830,9 +830,6 @@
 THIS_AD
 
 # good enough
-THREAD_INDEX_HEX
-
-# good enough
 TO_EQ_FM_DIRECT_MX
 
 # tflags net
@@ -887,28 +884,28 @@
 TW_GIBBERISH_MANY
 
 # good enough
-ADVANCE_FEE_3_NEW_FRM_MNY
+BIGNUM_EMAILS
 
 # good enough
-ADVANCE_FEE_4_NEW_FRM_MNY
+BODY_SINGLE_URI
 
 # good enough
-ADVANCE_FEE_5_NEW
+CK_HELO_GENERIC
 
 # good enough
-ANY_PILL_PRICE
+FILL_THIS_FORM_LOAN
 
 # good enough
-AXB_X_FF_SEZ_S
+HK_NAME_MR_MRS
 
 # good enough
-FILL_THIS_FORM_LONG
+LIST_PARTIAL_SHORT_MSG
 
 # good enough
-MONEY_BARRISTER
+MONEY_FORM
 
 # good enough
-TO_NO_BRKTS_FROM_MSSP
+MONEY_FROM_MISSP
 
 # tflags publish
 UC_GIBBERISH_OBFU
+ echo 'Committing promotions in rules/active.list...'
Committing promotions in rules/active.list...
+ svn commit -m 'promotions validated' rules/active.list
Sendingrules/active.list
Transmitting file data .done
Committing transaction...
Committed revision 1816412.
+ /usr/bin/perl masses/rule-qa/list-bad-rules
++ date +%w
+ [[ 1 = 3 ]]
+ for VER in '$VERSIONS'
+ make_tarball_for_version 3.4.2
+ version=3.4.2
+ tmpdir=/usr/local/spamassassin/automc/tmp/stage/3.4.2
+ rm -rf /usr/local/spamassassin/automc/tmp/stage/3.4.2
+ mkdir -p /usr/local/spamassassin/automc/tmp/stage/3.4.2
+ make clean
rm -f \
  SpamAssassin.bso SpamAssassin.def \
  SpamAssassin.exp SpamAssassin.x \
   blib/arch/auto/Mail/SpamAssassin/extralibs.all \
  blib/arch/auto/Mail/SpamAssassin/extralibs.ld Makefile.aperl \
  *.a *.o \
  *perl.core MYMETA.json \
  MYMETA.yml blibdirs.ts \
  core core.*perl.*.? \
  core.[0-9] core.[0-9][0-9] \
  core.[0-9][0-9][0-9] core.[0-9][0-9][0-9][0-9] \
  core.[0-9][0-9][0-9][0-9][0-9] libSpamAssassin.def \
  mon.out perl \
  perl perl.exe \
  perlmain.c pm_to_blib \
  pm_to_blib.ts so_locations \
  tmon.out 
rm -rf \
  *.cache blib \
  doc pod2htm* \
  qmail rules/*.pm \
  rules/70_inactive.cf sa-awl \
  sa-check_spamd sa-compile \
  sa-learn sa-update \
  spamassassin spamc/*.cache \
  spamc/*.o* spamc/*.so \
  spamc/Makefile spamc/config.h \
  spamc/config.log spamc/config.status \
  spamc/qmail-spamc spamc/replace/*.o* \
  spamc/spamc spamc/spamc.h \
  spamc/version.h spamd/*spamc* \
  spamd/spamd t/bayessql.cf \
  t/do_net t/log \
  t/sql_based_whitelist.cf version.env 
mv Makefile Makefile.old > /dev/null 2>&1
+ /usr/bin/perl Makefile.PL 
PREFIX=/usr/local/spamassassin/automc/tmp/stage/3.4.2
What email address or URL should be used in the suspected-spam report
text for users who want more information on your filter installation?
(In particular, ISPs should change this to a local Postmaster contact)
default text: [the administrator of that system] the administrator of that 
system

NOTE: settings for "make test" are now controlled using "t/config.dist". 
See that file if you wish to customize what tests are run, and how.

checking module dependencies and their versions...

***
NOTE: the optional Digest::SHA1 module is not installed.

  The Digest::SHA1 module is still required by the Razor2 plugin.
  Other modules prefer Digest::SHA, which is a Perl base module.

checking binary dependencies and their versions...

***
NOTE: the optional fetch binary is not installed.

   Sa-update will use curl, wget or fetch to download updates.  
   Because perl module LWP does not support IPv6, sa-update as of
   3.4.0 will use these standard programs to download rule updates
   leaving LWP as a fallback if none of the programs are found.

   *IMPORTANT NOTE*: You only need one of these programs 
   It's only a concern if you are warned about all 3 
   i.e. (c

Re: NOTE: Warning to Abusers of Update Servers

2017-11-27 Thread Kevin A. McGrail

On 11/27/2017 12:06 AM, Dave Warren wrote:
I’m not currently behind CloudFlare, but I already wrote code to purge 
their cache whenever mirrored content is rsync’d in case I do move 
anything under CloudFlare in the future, or use any other CDN. I’m 
automating a couple mirrors to flip to CloudFlare when there is a 
spike, but I have not enabled this code for SpamAssassin.


Are there cases where any files are updated other than the MIRROR* 
files? Or does this mirror only add files? Basically I’m wondering if 
I should dump the entire cache or just these specific files?
Because the items are release artifacts, they are never altered or 
removed, just added.


If you can have a < 10 min cache on these files, that would be fine

GPG.KEY
index.html
MIRROR.CHECK
MIRRORED.BY
robots.txt


Regards,
KAM


[Bug 7508] New: Suboptimal permissions of mirrored rulesets files (on sa-update mirrors)

2017-11-27 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7508

Bug ID: 7508
   Summary: Suboptimal permissions of mirrored rulesets files (on
sa-update mirrors)
   Product: Spamassassin
   Version: unspecified
  Hardware: All
OS: Linux
Status: NEW
  Severity: minor
  Priority: P2
 Component: sysadmins
  Assignee: sysadmins@spamassassin.apache.org
  Reporter: jens.schleuse...@t-online.de
  Target Milestone: Undefined

The mirrored rulesets tarball files (and the according ASC and SHA1 files) have
odd and heterogeneous permissions. Here an extract:

 -rw-rw-r--  11 Nov 27 14:17 MIRROR.CHECK
 -rw-r--r-- 100 Nov 27 09:31 1816413.tar.gz.sha1
 -rw-r--r-- 819 Nov 27 09:31 1816413.tar.gz.asc
 -rw-r--r--  207813 Nov 27 09:31 1816413.tar.gz
 -r-xr--r-- 819 Nov 27 03:55 1816372.tar.gz.asc
 -r-xr--r-- 113 Nov 27 03:55 1816372.tar.gz.sha1
 -r-xr--r--  275070 Nov 27 03:55 1816372.tar.gz
 -rw-r--r--1309 Nov 26 22:30 MIRRORED.BY
 ...
 -rw-rw-r-- 100 Jun  4 10:30 1797561.tar.gz.sha1
 -rw-rw-r-- 819 Jun  4 10:30 1797561.tar.gz.asc
 -rw-rw-r--  206546 Jun  4 10:30 1797561.tar.gz
 ...
 -rwxrwxr-x  56 Feb 15  2007 507739.tar.gz.sha1
 -rwxrwxr-x  126897 Feb 15  2007 507739.tar.gz
 -rwxrwxr-x 823 Feb 15  2007 507739.tar.gz.asc

In my eyes the execution-flags are totally wrong. And also the write-flags are
superfluous and at least theoretically a little bit dangerous. I assume the
mirroring ("rsync") will even work without a write-flag for the owner.

But as Kevin A. McGrail has written in the sysadmins mailing list it seems not
a big problem but only a flaw "because rules are crypto signed".

-- 
You are receiving this mail because:
You are the assignee for the bug.

[Bug 7508] Suboptimal permissions of mirrored rulesets files (on sa-update mirrors)

2017-11-27 Thread bugzilla-daemon
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7508

Dave Jones  changed:

   What|Removed |Added

 CC||da...@apache.org
 Status|NEW |ASSIGNED
   Assignee|sysadm...@spamassassin.apac |da...@apache.org
   |he.org  |

--- Comment #1 from Dave Jones  ---
You are correct.  The perms are not optimal.  However, web servers shouldn't be
executing any of these file types as they are not scripts or executable files.

I will look at the scripts that generate the rulesets and set perms to 444. 
Rsyncs should be running as root on the mirrors so this should not impact
rsync'ing.

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: NOTE: Warning to Abusers of Update Servers

2017-11-27 Thread Jari Fredriksson


> Kevin A. McGrail  kirjoitti 27.11.2017 kello 14.22:
> 
> On 11/27/2017 12:06 AM, Dave Warren wrote:
>> I’m not currently behind CloudFlare, but I already wrote code to purge their 
>> cache whenever mirrored content is rsync’d in case I do move anything under 
>> CloudFlare in the future, or use any other CDN. I’m automating a couple 
>> mirrors to flip to CloudFlare when there is a spike, but I have not enabled 
>> this code for SpamAssassin.
>> 
>> Are there cases where any files are updated other than the MIRROR* files? Or 
>> does this mirror only add files? Basically I’m wondering if I should dump 
>> the entire cache or just these specific files?
> Because the items are release artifacts, they are never altered or removed, 
> just added.
> 
> If you can have a < 10 min cache on these files, that would be fine
> 
> GPG.KEY
> index.html
> MIRROR.CHECK
> MIRRORED.BY
> robots.txt
> 
> 
> Regards,
> KAM

I would be a poor SysAdmin :( the second mirror I added was not properly 
configured… Now it is, but of course I seem to be commented out because of this.

br. jarif


signature.asc
Description: Message signed with OpenPGP


Re: NOTE: Warning to Abusers of Update Servers

2017-11-27 Thread Dave Jones

On 11/27/2017 02:19 PM, Jari Fredriksson wrote:

Kevin A. McGrail  kirjoitti 27.11.2017 kello 14.22:

On 11/27/2017 12:06 AM, Dave Warren wrote:

I’m not currently behind CloudFlare, but I already wrote code to purge their 
cache whenever mirrored content is rsync’d in case I do move anything under 
CloudFlare in the future, or use any other CDN. I’m automating a couple mirrors 
to flip to CloudFlare when there is a spike, but I have not enabled this code 
for SpamAssassin.

Are there cases where any files are updated other than the MIRROR* files? Or 
does this mirror only add files? Basically I’m wondering if I should dump the 
entire cache or just these specific files?

Because the items are release artifacts, they are never altered or removed, 
just added.

If you can have a < 10 min cache on these files, that would be fine

GPG.KEY
index.html
MIRROR.CHECK
MIRRORED.BY
robots.txt


Regards,
KAM

I would be a poor SysAdmin :( the second mirror I added was not properly 
configured… Now it is, but of course I seem to be commented out because of this.

br. jarif


It's live again.  My bandwidth usage has dropped from about 65 GB a day 
to about 42 GB yesterday.  All of these new mirrors are helping out a lot.