checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

2018-01-17 Thread root 
Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/ (78.47.167.123): UP (CURRENT)

http://www.sa-update.pccc.com/ (69.171.29.39): UP (CURRENT)

http://sa-update.secnap.net/ (204.89.241.6): UP (CURRENT)

http://sa-update.space-pro.be/ (176.28.55.20): UP (CURRENT)

http://sa-update.ena.com/ (96.5.1.5): DOWN

http://sa-update.razx.cloud/ (104.27.152.42): UP (CURRENT)

http://sa-update.razx.cloud/ (104.27.153.42): UP (CURRENT)

http://sa-update.fossies.org/ (138.201.17.217): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.124.130): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.212.58): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.120.157): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.74.190): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.75.190): UP (CURRENT)

http://sa-update.spamassassin.org/ (64.142.56.146): UP (CURRENT)

Cron ~/svn/trunk/build/mkupdates/run_nightly | /usr/bin/tee /var/www/automc.spamassassin.org/mkupdates/mkupdates.txt

2018-01-17 Thread Cron Daemon 
+ promote_active_rules
+ pwd
+ /usr/bin/perl build/mkupdates/listpromotable
/usr/local/spamassassin/automc/svn/trunk
HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/3-days-ago?xml=1
+ mv rules/active.list.new rules/active.list
+ svn diff rules
+ cat /var/www/ruleqa.spamassassin.org/reports/LATEST
Index: rules/active.list
===
--- rules/active.list   (revision 1821222)
+++ rules/active.list   (working copy)
@@ -59,9 +59,6 @@
 ADVANCE_FEE_4_NEW
 
 # good enough
-ADVANCE_FEE_4_NEW_MONEY
-
-# good enough
 ADVANCE_FEE_5_NEW_FRM_MNY
 
 # good enough
@@ -74,6 +71,9 @@
 ALL_TRUSTED
 
 # good enough
+APOSTROPHE_FROM
+
+# good enough
 APOSTROPHE_TOCC
 
 # good enough
@@ -254,9 +254,6 @@
 FROM_MISSPACED
 
 # good enough
-FROM_MISSP_DYNIP
-
-# good enough
 FROM_MISSP_FREEMAIL
 
 # good enough
@@ -263,9 +260,6 @@
 FROM_MISSP_MSFT
 
 # good enough
-FROM_MISSP_PHISH
-
-# good enough
 FROM_MISSP_REPLYTO
 
 # tflags net
@@ -296,6 +290,9 @@
 FSL_HELO_BARE_IP_1
 
 # good enough
+FSL_HELO_DEVICE
+
+# good enough
 FSL_HELO_FAKE
 
 # tflags publish
@@ -389,9 +386,6 @@
 HK_NAME_DRUGS
 
 # good enough
-HK_RANDOM_ENVFROM
-
-# good enough
 HK_SCAM_N2
 
 # tflags userconf
@@ -418,6 +412,9 @@
 # tflags publish
 LUCRATIVE
 
+# good enough
+MANY_PILL_PRICE
+
 # tflags publish
 MANY_SPAN_IN_TEXT
 
@@ -830,6 +827,9 @@
 TO_NO_BRKTS_PCNT
 
 # good enough
+TVD_PH_BODY_META
+
+# good enough
 TVD_QUAL_MEDS
 
 # good enough
@@ -851,35 +851,29 @@
 TW_GIBBERISH_MANY
 
 # good enough
-ADVANCE_FEE_4_NEW_FRM_MNY
+BODY_SINGLE_URI
 
 # good enough
-ADVANCE_FEE_5_NEW
+FILL_THIS_FORM_LOAN
 
 # good enough
-COMPENSATION
+HELO_LH_HOME
 
 # good enough
-FILL_THIS_FORM_LONG
+LIST_PARTIAL_SHORT_MSG
 
 # good enough
-HDRS_LCASE
+MANY_HDRS_LCASE
 
 # good enough
-MANY_PILL_PRICE
+MONEY_FORM
 
 # good enough
-MIMEOLE_DIRECT_TO_MX
+MONEY_FROM_MISSP
 
 # good enough
-MSGID_NOFQDN1
+XFER_LOTSA_MONEY
 
-# good enough
-TO_NO_BRKTS_FROM_MSSP
-
-# good enough
-TVD_PH_BODY_META
-
 # tflags publish
 UC_GIBBERISH_OBFU
 
+ echo 'Committing promotions in rules/active.list...'
+ svn commit -m 'promotions validated' rules/active.list
Committing promotions in rules/active.list...
Sendingrules/active.list
Transmitting file data .done
Committing transaction...
Committed revision 1821350.
+ /usr/bin/perl masses/rule-qa/list-bad-rules
++ date +%w
+ [[ 3 = 3 ]]
+ echo 'From: nore...@sa-vm1.apache.org (Rules Report Cron)'
+ echo 'Subject: [auto] bad sandbox rules report'
+ echo
+ cat /var/www/ruleqa.spamassassin.org/reports/badrules.txt
+ /usr/sbin/sendmail -oi d...@spamassassin.apache.org
+ for VER in '$VERSIONS'
+ make_tarball_for_version 3.4.2
+ version=3.4.2
+ tmpdir=/usr/local/spamassassin/automc/tmp/stage/3.4.2
+ rm -rf /usr/local/spamassassin/automc/tmp/stage/3.4.2
+ mkdir -p /usr/local/spamassassin/automc/tmp/stage/3.4.2
+ make clean
rm -f \
  SpamAssassin.bso SpamAssassin.def \
  SpamAssassin.exp SpamAssassin.x \
   blib/arch/auto/Mail/SpamAssassin/extralibs.all \
  blib/arch/auto/Mail/SpamAssassin/extralibs.ld Makefile.aperl \
  *.a *.o \
  *perl.core MYMETA.json \
  MYMETA.yml blibdirs.ts \
  core core.*perl.*.? \
  core.[0-9] core.[0-9][0-9] \
  core.[0-9][0-9][0-9] core.[0-9][0-9][0-9][0-9] \
  core.[0-9][0-9][0-9][0-9][0-9] libSpamAssassin.def \
  mon.out perl \
  perl perl.exe \
  perlmain.c pm_to_blib \
  pm_to_blib.ts so_locations \
  tmon.out 
rm -rf \
  *.cache blib \
  doc pod2htm* \
  qmail rules/*.pm \
  rules/70_inactive.cf sa-awl \
  sa-check_spamd sa-compile \
  sa-learn sa-update \
  spamassassin spamc/*.cache \
  spamc/*.o* spamc/*.so \
  spamc/Makefile spamc/config.h \
  spamc/config.log spamc/config.status \
  spamc/qmail-spamc spamc/replace/*.o* \
  spamc/spamc spamc/spamc.h \
  spamc/version.h spamd/*spamc* \
  spamd/spamd t/bayessql.cf \
  t/do_net t/log \
  t/sql_based_whitelist.cf version.env 
mv Makefile Makefile.old > /dev/null 2>&1
+ /usr/bin/perl Makefile.PL 
PREFIX=/usr/local/spamassassin/automc/tmp/stage/3.4.2
What email address or URL should be used in the suspected-spam report
text for users who want more information on your filter installation?
(In particular, ISPs should change this to a local Postmaster contact)
default text: [the administrator of that system] the administrator of that 
system

NOTE: settings for "make test" are now controlled using "t/config.dist". 
See that file if you wish to customize what tests are run, and how.

checking module dependencies and their versions...

***
NOTE: the optional Digest::SHA1 module is not installed.

  The Digest::SHA1 module is still required by the Razor2 plugin.
  Other modules prefer Digest::SHA, which is a Perl base module.

checking binary dependencies and their versions...

***

checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

2018-01-17 Thread root 
Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/ (78.47.167.123): UP (CURRENT)

http://www.sa-update.pccc.com/ (69.171.29.39): UP (CURRENT)

http://sa-update.secnap.net/ (204.89.241.6): UP (CURRENT)

http://sa-update.space-pro.be/ (176.28.55.20): UP (CURRENT)

http://sa-update.ena.com/ (96.5.1.5): DOWN

http://sa-update.razx.cloud/ (104.27.152.42): UP (CURRENT)

http://sa-update.razx.cloud/ (104.27.153.42): UP (CURRENT)

http://sa-update.fossies.org/ (138.201.17.217): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.120.157): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.124.130): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.212.58): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.75.190): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.74.190): UP (CURRENT)

http://sa-update.spamassassin.org/ (64.142.56.146): UP (CURRENT)

checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

2018-01-17 Thread root 
Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/ (78.47.167.123): UP (CURRENT)

http://www.sa-update.pccc.com/ (69.171.29.39): UP (CURRENT)

http://sa-update.secnap.net/ (204.89.241.6): UP (CURRENT)

http://sa-update.space-pro.be/ (176.28.55.20): UP (CURRENT)

http://sa-update.ena.com/ (96.5.1.5): DOWN

http://sa-update.razx.cloud/ (104.27.152.42): UP (CURRENT)

http://sa-update.razx.cloud/ (104.27.153.42): UP (CURRENT)

http://sa-update.fossies.org/ (138.201.17.217): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.120.157): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.29.131): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.212.58): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.75.190): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.74.190): UP (CURRENT)

http://sa-update.spamassassin.org/ (64.142.56.146): UP (CURRENT)

checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

2018-01-17 Thread root 
Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/ (78.47.167.123): UP (CURRENT)

http://www.sa-update.pccc.com/ (69.171.29.39): UP (CURRENT)

http://sa-update.secnap.net/ (204.89.241.6): UP (CURRENT)

http://sa-update.space-pro.be/ (176.28.55.20): UP (CURRENT)

http://sa-update.ena.com/ (96.4.1.5): UP (CURRENT)

http://sa-update.ena.com/ (96.5.1.5): DOWN

http://sa-update.razx.cloud/ (104.27.152.42): UP (CURRENT)

http://sa-update.razx.cloud/ (104.27.153.42): UP (CURRENT)

http://sa-update.fossies.org/ (138.201.17.217): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.29.131): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.212.58): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.120.157): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.74.190): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.75.190): UP (CURRENT)

http://sa-update.spamassassin.org/ (64.142.56.146): UP (CURRENT)

checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

2018-01-17 Thread root 
Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/ (78.47.167.123): UP (CURRENT)

http://www.sa-update.pccc.com/ (69.171.29.39): UP (CURRENT)

http://sa-update.secnap.net/ (204.89.241.6): UP (CURRENT)

http://sa-update.space-pro.be/ (176.28.55.20): UP (CURRENT)

http://sa-update.ena.com/ (96.5.1.5): DOWN

http://sa-update.razx.cloud/ (104.27.152.42): UP (CURRENT)

http://sa-update.razx.cloud/ (104.27.153.42): UP (CURRENT)

http://sa-update.fossies.org/ (138.201.17.217): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.212.58): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.124.130): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.29.131): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.75.190): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.74.190): UP (CURRENT)

http://sa-update.spamassassin.org/ (64.142.56.146): UP (CURRENT)

checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

2018-01-17 Thread root 
Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/ (78.47.167.123): UP (CURRENT)

http://www.sa-update.pccc.com/ (69.171.29.39): UP (CURRENT)

http://sa-update.secnap.net/ (204.89.241.6): UP (CURRENT)

http://sa-update.space-pro.be/ (176.28.55.20): UP (CURRENT)

http://sa-update.ena.com/ (96.4.1.5): UP (CURRENT)

http://sa-update.ena.com/ (96.5.1.5): DOWN

http://sa-update.razx.cloud/ (104.27.153.42): UP (CURRENT)

http://sa-update.razx.cloud/ (104.27.152.42): UP (CURRENT)

http://sa-update.fossies.org/ (138.201.17.217): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.212.58): UP (CURRENT)

http://sa-update.verein-clean.net/ (148.251.29.131): UP (CURRENT)

http://sa-update.verein-clean.net/ (37.252.120.157): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.75.190): UP (CURRENT)

http://sa-update.bitwell.fi/ (104.31.74.190): UP (CURRENT)

http://sa-update.spamassassin.org/ (64.142.56.146): UP (CURRENT)

Re: Notes about Trap - Re: colo/incoming.spamassassin.org server

2018-01-17 Thread Kevin A. McGrail 
I talked to Justin and he is not aware of a SurfNet machine.  They are a 
targeted sponsor who has helped us.  I've sent one email seeing if I can 
dig anything up but I must concur.


Dave's research shows that "trap-proc.spamassassin.org and 
spam-upload.spamassassin.org both resolve to 192.87.106.247 which is a 
SURFnet IP in the Netherlands.  I am not able to ping or nmap any 
response to that IP."


With that, we are giving up on the mass-check centralized box and 
shutting down the trap-proc and sought processing.



These are the tasks:

1 - Find out more from SurfNet about the 192.87.106.247 machine [KAM]

2 - [Joe, can you help?] Find out what Sonic is trapping and where they 
are sending it.


3 - Look at Sought 2.0 [Perhaps with AXB and Dave Jones]

4 - Delete the backup data on sa-vm1, move /usr/local/spamassassin to a 
different mount so that entire partition can be returned to Infra [DAVE]


5 - Open Infra ticket to return data. [KAM]


So Joe can you do the following:

- Please deprecate the old colo box.  Thank you for hosting and 
providing it on behalf of the project!


- the ASF would like to recognize Sonic as a sponsor.  Any update on the 
paperwork?  If paperwork is an issue, I can grandfather you without it.


- How's our bandwidth usage look to Sonic.  Should we bump it up/down?


Dave:

Any chance you can handle these tasks or let me know if they already 
were done?


- Put an A record for the box into DNS for apachesf.Spamassassin.org

- Add to SysAdmins Docs / wiki the new apachesf and remove colo and note 
that trap-proc is no longer online.



On 1/16/2018 7:19 PM, Dave Jones wrote:

On 01/16/2018 09:00 AM, Kevin A. McGrail wrote:

Joe and Dave + SASA:

I have combed my notes.  Here's what I have which I have removed some 
password info on but I think it can help rebuild the process.  Dave, 
can you take a look?  I can get you passwords. Talon1 still exists, 
spamassassin-vm does not, I have a backup of spamassassin-vm from 3.7 
months ago.


Regards,
KAM

#1 - Some boxes are just names for other boxes
trap-proc.spamassassin.org. Sonic has scripts set up to archive 
collected spam to that server.





trap-proc.spamassassin.org and spam-upload.spamassassin.org both 
resolve to 192.87.106.247 which is a SURFnet IP in the Netherlands.  I 
am not able to ping or nmap any response to that IP.




#2 - My notes from spamassassin-vm.apache.org that catastrophically 
died:


this was the traps cron that needs to be added on spamassassin-vm

20 2 * * * rsync -rze ssh --whole-file --size-only --delete 
j...@trap-proc.spamassassin.org 
:/home/jm/cor/. 
/export/home/bbmass/uploadedcorpora/traps/.


DONE - add this traps account
DONE - fix perms for /export/home/bbmass/uploadedcorpora/traps/
DONE - add cron job




Since this IP is not responding to anything, not sure how long this 
has been offline.  I can't see what purpose this would be be doing 
based on the masscheck flow that worked on last May on sa-vm1.apache.org.


The old spamassassin-vm.apache.org used to run a buildbot version for 
on-demand masschecking of uploaded corpus but we don't have that setup 
again anywhere.  If we do set this up again, I don't want to put this 
on the sa-vm1.apache.org without any swap space. I don't see the value 
of uploading ham/spam and running the masscheck centrally anymore.




#3 - From April 2017

Let me know if you are not the correct person to talk to about this, but
we are having issues reaching trap-proc.spamassassin.org. It looks like
we have some scripts set up to archive collected spam to that server,
and I haven't seen a successful connection for a few days now.

--
Grant Keller
System Operations
grant.kel...@sonic.com 


#4 - from 2014


The box at Sonic is the backend for the SpamAssassin spamtraps feed. 
 To be honest, I am not sure anyone or anything is consuming the 
collected data at this stage -- it should probably be shut down, 
unless someone wants to take it over?




My vote is to shut it down.



incoming.spamassassin.org : this is the spamtrap machine at Sonic. 
  Basically, qpsmtpd
handles the incoming SMTP traffic, handing it off via a Gearman queue 
to "gears" -- a
set of scripts running in the background which filter out noise, 
crap, bounces, etc.

then buffer them to mbox files and upload.

/home/trap contains the code, /home/trapper is the output files. 
  /etc/init.d/gears starts the
scripts which compose it, copying them to /tmpfs first so they don't 
hit the disk where possible,

for speed.

The main config file is at /home/trap/code/gears/config .

The buffered mbox files are then uploaded to my S3 account, using an 
IAM credential which can only
access one single bucket called "mailtrap".  After 1 day those files 
are auto-expired.


This stuff all appears to be working ok, although the volume is 
pretty high (and I suspect

it's costing me a fair bit of money even despite the auto-expirat

Re: Notes about Trap - Re: colo/incoming.spamassassin.org server

2018-01-17 Thread Dave Jones 

On 01/17/2018 10:13 AM, Kevin A. McGrail wrote:
I talked to Justin and he is not aware of a SurfNet machine.  They are a 
targeted sponsor who has helped us.  I've sent one email seeing if I can 
dig anything up but I must concur.


Dave's research shows that "trap-proc.spamassassin.org and 
spam-upload.spamassassin.org both resolve to 192.87.106.247 which is a 
SURFnet IP in the Netherlands.  I am not able to ping or nmap any 
response to that IP."


With that, we are giving up on the mass-check centralized box and 
shutting down the trap-proc and sought processing.



These are the tasks:

1 - Find out more from SurfNet about the 192.87.106.247 machine [KAM]

2 - [Joe, can you help?] Find out what Sonic is trapping and where they 
are sending it.


3 - Look at Sought 2.0 [Perhaps with AXB and Dave Jones]

4 - Delete the backup data on sa-vm1, move /usr/local/spamassassin to a 
different mount so that entire partition can be returned to Infra [DAVE]




DJ - I will take this task.


5 - Open Infra ticket to return data. [KAM]



DJ - I will take this task.



So Joe can you do the following:

- Please deprecate the old colo box.  Thank you for hosting and 
providing it on behalf of the project!


- the ASF would like to recognize Sonic as a sponsor.  Any update on the 
paperwork?  If paperwork is an issue, I can grandfather you without it.


- How's our bandwidth usage look to Sonic.  Should we bump it up/down?


Dave:

Any chance you can handle these tasks or let me know if they already 
were done?


- Put an A record for the box into DNS for apachesf.Spamassassin.org



DJ - Already done a couple of weeks ago.

- Add to SysAdmins Docs / wiki the new apachesf and remove colo and note 
that trap-proc is no longer online.




DJ - Done.




On 1/16/2018 7:19 PM, Dave Jones wrote:

On 01/16/2018 09:00 AM, Kevin A. McGrail wrote:

Joe and Dave + SASA:

I have combed my notes.  Here's what I have which I have removed some 
password info on but I think it can help rebuild the process.  Dave, 
can you take a look?  I can get you passwords. Talon1 still exists, 
spamassassin-vm does not, I have a backup of spamassassin-vm from 3.7 
months ago.


Regards,
KAM

#1 - Some boxes are just names for other boxes
trap-proc.spamassassin.org. Sonic has scripts set up to archive 
collected spam to that server.





trap-proc.spamassassin.org and spam-upload.spamassassin.org both 
resolve to 192.87.106.247 which is a SURFnet IP in the Netherlands.  I 
am not able to ping or nmap any response to that IP.




#2 - My notes from spamassassin-vm.apache.org that catastrophically 
died:


this was the traps cron that needs to be added on spamassassin-vm

20 2 * * * rsync -rze ssh --whole-file --size-only --delete 
j...@trap-proc.spamassassin.org 
:/home/jm/cor/. 
/export/home/bbmass/uploadedcorpora/traps/.


DONE - add this traps account
DONE - fix perms for /export/home/bbmass/uploadedcorpora/traps/
DONE - add cron job




Since this IP is not responding to anything, not sure how long this 
has been offline.  I can't see what purpose this would be be doing 
based on the masscheck flow that worked on last May on sa-vm1.apache.org.


The old spamassassin-vm.apache.org used to run a buildbot version for 
on-demand masschecking of uploaded corpus but we don't have that setup 
again anywhere.  If we do set this up again, I don't want to put this 
on the sa-vm1.apache.org without any swap space. I don't see the value 
of uploading ham/spam and running the masscheck centrally anymore.




#3 - From April 2017

Let me know if you are not the correct person to talk to about this, but
we are having issues reaching trap-proc.spamassassin.org. It looks like
we have some scripts set up to archive collected spam to that server,
and I haven't seen a successful connection for a few days now.

--
Grant Keller
System Operations
grant.kel...@sonic.com 


#4 - from 2014


The box at Sonic is the backend for the SpamAssassin spamtraps feed. 
 To be honest, I am not sure anyone or anything is consuming the 
collected data at this stage -- it should probably be shut down, 
unless someone wants to take it over?




My vote is to shut it down.



incoming.spamassassin.org : this is the spamtrap machine at Sonic. 
  Basically, qpsmtpd
handles the incoming SMTP traffic, handing it off via a Gearman queue 
to "gears" -- a
set of scripts running in the background which filter out noise, 
crap, bounces, etc.

then buffer them to mbox files and upload.

/home/trap contains the code, /home/trapper is the output files. 
  /etc/init.d/gears starts the
scripts which compose it, copying them to /tmpfs first so they don't 
hit the disk where possible,

for speed.

The main config file is at /home/trap/code/gears/config .

The buffered mbox files are then uploaded to my S3 account, using an 
IAM credential which can only
access one single bucket called "mailtrap".  After 1 day those files 
are a

[Bug 7505] build/mkupdates/listpromotable deprecated goto

2018-01-17 Thread bugzilla-daemon 
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7505

Dave Jones  changed:

   What|Removed |Added

 CC||da...@apache.org

--- Comment #1 from Dave Jones  ---
This one is over my head since I don't have a lot of experience with perl.  I
think the goto needs to be replaced with function calls but not sure.

-- 
You are receiving this mail because:
You are the assignee for the bug.

Re: Notes about Trap - Re: colo/incoming.spamassassin.org server

2018-01-17 Thread Kevin A. McGrail 

ty!

On 1/17/2018 11:53 AM, Dave Jones wrote:

On 01/17/2018 10:13 AM, Kevin A. McGrail wrote:
I talked to Justin and he is not aware of a SurfNet machine.  They 
are a targeted sponsor who has helped us.  I've sent one email seeing 
if I can dig anything up but I must concur.


Dave's research shows that "trap-proc.spamassassin.org and 
spam-upload.spamassassin.org both resolve to 192.87.106.247 which is 
a SURFnet IP in the Netherlands.  I am not able to ping or nmap any 
response to that IP."


With that, we are giving up on the mass-check centralized box and 
shutting down the trap-proc and sought processing.



These are the tasks:

1 - Find out more from SurfNet about the 192.87.106.247 machine [KAM]

2 - [Joe, can you help?] Find out what Sonic is trapping and where 
they are sending it.


3 - Look at Sought 2.0 [Perhaps with AXB and Dave Jones]

4 - Delete the backup data on sa-vm1, move /usr/local/spamassassin to 
a different mount so that entire partition can be returned to Infra 
[DAVE]




DJ - I will take this task.


5 - Open Infra ticket to return data. [KAM]



DJ - I will take this task.



So Joe can you do the following:

- Please deprecate the old colo box.  Thank you for hosting and 
providing it on behalf of the project!


- the ASF would like to recognize Sonic as a sponsor.  Any update on 
the paperwork?  If paperwork is an issue, I can grandfather you 
without it.


- How's our bandwidth usage look to Sonic.  Should we bump it up/down?


Dave:

Any chance you can handle these tasks or let me know if they already 
were done?


- Put an A record for the box into DNS for apachesf.Spamassassin.org



DJ - Already done a couple of weeks ago.

- Add to SysAdmins Docs / wiki the new apachesf and remove colo and 
note that trap-proc is no longer online.




DJ - Done.




On 1/16/2018 7:19 PM, Dave Jones wrote:

On 01/16/2018 09:00 AM, Kevin A. McGrail wrote:

Joe and Dave + SASA:

I have combed my notes.  Here's what I have which I have removed 
some password info on but I think it can help rebuild the process.  
Dave, can you take a look?  I can get you passwords. Talon1 still 
exists, spamassassin-vm does not, I have a backup of 
spamassassin-vm from 3.7 months ago.


Regards,
KAM

#1 - Some boxes are just names for other boxes
trap-proc.spamassassin.org. Sonic has scripts set up to archive 
collected spam to that server.





trap-proc.spamassassin.org and spam-upload.spamassassin.org both 
resolve to 192.87.106.247 which is a SURFnet IP in the Netherlands.  
I am not able to ping or nmap any response to that IP.




#2 - My notes from spamassassin-vm.apache.org that catastrophically 
died:


this was the traps cron that needs to be added on spamassassin-vm

20 2 * * * rsync -rze ssh --whole-file --size-only --delete 
j...@trap-proc.spamassassin.org 
:/home/jm/cor/. 
/export/home/bbmass/uploadedcorpora/traps/.


DONE - add this traps account
DONE - fix perms for /export/home/bbmass/uploadedcorpora/traps/
DONE - add cron job




Since this IP is not responding to anything, not sure how long this 
has been offline.  I can't see what purpose this would be be doing 
based on the masscheck flow that worked on last May on 
sa-vm1.apache.org.


The old spamassassin-vm.apache.org used to run a buildbot version 
for on-demand masschecking of uploaded corpus but we don't have that 
setup again anywhere.  If we do set this up again, I don't want to 
put this on the sa-vm1.apache.org without any swap space. I don't 
see the value of uploading ham/spam and running the masscheck 
centrally anymore.




#3 - From April 2017

Let me know if you are not the correct person to talk to about 
this, but
we are having issues reaching trap-proc.spamassassin.org. It looks 
like

we have some scripts set up to archive collected spam to that server,
and I haven't seen a successful connection for a few days now.

--
Grant Keller
System Operations
grant.kel...@sonic.com 


#4 - from 2014


The box at Sonic is the backend for the SpamAssassin spamtraps 
feed.  To be honest, I am not sure anyone or anything is consuming 
the collected data at this stage -- it should probably be shut 
down, unless someone wants to take it over?




My vote is to shut it down.



incoming.spamassassin.org : this is the spamtrap machine at Sonic. 
  Basically, qpsmtpd
handles the incoming SMTP traffic, handing it off via a Gearman 
queue to "gears" -- a
set of scripts running in the background which filter out noise, 
crap, bounces, etc.

then buffer them to mbox files and upload.

/home/trap contains the code, /home/trapper is the output files. 
  /etc/init.d/gears starts the
scripts which compose it, copying them to /tmpfs first so they 
don't hit the disk where possible,

for speed.

The main config file is at /home/trap/code/gears/config .

The buffered mbox files are then uploaded to my S3 account, using 
an IAM credential which can only
access one single buc

Re: Notes about Trap - Re: colo/incoming.spamassassin.org server

2018-01-17 Thread Joe Muller 
On 01/17/2018 08:13 AM, Kevin A. McGrail wrote:
> 2 - [Joe, can you help?] Find out what Sonic is trapping and where
> they are sending it.

   Our setup was/is as follows:
- a server running qsmtpd and a customized version of sendmail is at
mailin.sonic.net (157.131.0.10)
- the messages that hit our spam traps are delivered to this host
- a subset of messages are placed in a maildir-format mailbox, which is
archived into a tarball nightly (filename includes timestamp)
- the resulting tarball is SCP'ed to
scpso...@trap-proc.spamassassin.org:spam.[TIMESTAMP].tar

   Please let me know where you'd like the trapped mail to be copied to
and how it should be transferred.

> So Joe can you do the following:
>
> - Please deprecate the old colo box.  Thank you for hosting and
> providing it on behalf of the project!

   I've kicked off a turn-down of the old server at 76.191.162.2. I'm
glad we got you guys upgraded to newer, faster hardware!

> - the ASF would like to recognize Sonic as a sponsor.  Any update on
> the paperwork?  If paperwork is an issue, I can grandfather you
> without it.

   If we can just be grandfathered in, that's preferable. I'll run the
paperwork up the chain here so we can be official. :)

> - How's our bandwidth usage look to Sonic.  Should we bump it up/down?

   At the moment, you're provisioned for a full 100Mbps symmetric. I've
attached a screenshot of the bandwidth graph for this month. Peak
utilization (5min sampling) is 11.6Mbps, with a total of about 450GB
moved over the wire to date. Your current uplink is served from a Cisco
3600-series switch, so traffic policing is simply 10 or 100Mbps (aka
line-rate).

   The latest bandwidth report and remote outlet control can be accessed
at https://members.sonic.net/colo-tools/. (At the moment, you'll need to
use the 'apachesf' login.)


   Let me know if I missed anything!


-- Joe Muller
Sonic System Operations