So it was only 3 years ago when SpamAssassin Zones died catastrophically
and was replaced by spamassassin-vm.
We are now replacing spamassassin-vm with sa-vm1. Unfortunately, the
box had to be turned off.
In the past, there were separate zones boxes because the grinding for
masscheck was taking too long and/or bringing the entire virtual machine
to it's knees.
In any case, these notes were what I wrote 3 years ago when rebuilding
that machine in case they are helpful as we rebuild again!
New Zones box
2014-04-24
spamassassin-vm.apache.org
DONE - 12 – Delete /files/old_files_2014-05-01.tar.bz2 if it is not
needed. it came from /home/corpus-rsync/ and is believe to be old logs
that are no longer needed but is quite a bit of data.
13 – is /home/automc’s .forward / procmail file needed? This I believe
is an email based masscheck I believe and I don't think it has worked
for ages - EMAILED JUSTIN
13a3 - Fix this issue in ruleqa.cgi [Wed Jun 25 01:06:08 2014]
ruleqa.cgi: Use of uninitialized value in substitution (s///) at
./masses/rule-qa/automc/ruleqa.cgi line 1997. This is putting a lot of
information in the freqs daemon log on zones2
/zonestorage/spamassassin2/home/automc/freqsd/log
13b - Get time to sync correctly for zones2. It's off by about 40
seconds as of 7/5/2014 - OPENED INFRA TICKET
https://issues.apache.org/jira/browse/INFRA-7956 - Fixed and ping gav on
5/4/15 to see what alternate port he used
13b2 - Disable the bb-jm cronjob. The cronjob exists with no account so
it generates a lot of errors and looks that way for 5 or so years -
Emailed Justin to confirm
22 - Get zones2 merged onto the box and deprecate zones2 and zones
permanently to simplify administration.
- Make a copy of zones to talonjr - rsync -e ssh
r...@spamassassin2.zones.apache.org
<mailto:r...@spamassassin2.zones.apache.org>:/
/mnt/das2/spamassassin2.zones.apache.org/backup --verbose --recursive
--relative --links --perms --owner --group --times --numeric-ids --stats
--progress --delete --exclude /proc --exclude /dev --exclude /home/jm
--exclude /home/felicity --exclude /home/mmartinec --exclude
/home/quinlan --exclude /home/automc --exclude /home/corpus-rsync
--exclude /home/buildbot --exclude /home/dos --exclude /home/bbmass
--exclude /home/traps | tee
/mnt/das2/spamassassin2.zones.apache.org/zones2-backup-rsync.log - IN
PROGRESS 5/26
- Make a backup of zones to sa-vm - rsync -e ssh
r...@spamassassin2.zones.apache.org
<mailto:r...@spamassassin2.zones.apache.org>:/ /root/sa2.zones-backup
--verbose --recursive --relative --links --perms --owner --group --times
--numeric-ids --stats --progress --delete | tee
~/zones2-backup-rsync.log - Pending the issue that there is possibly
310GB in /zonestorage
- remove from crashplan when live
22a - If I have a modern CentOS box on ASF infra, why do we need the
sonic CentOS box? To simplify the administration, I'd like to
consolidate but no one remembers why we have zones, zones2, incoming,
etc. and I don't want to revisit the same problems already solved ;-)
23 - update lots of documentation
SpamAssassinAsfZone
InfraNotes - Updated
RsyncConfig
ApacheSpamassassinWiki:SaUpdateBackend
RescoreMassCheck
AutoRuleUpdates - Good info about mass check
DONE - 27 - Get more CPU/Disk IO/RAM as needed for spamassassin-vm -
Emails sent to infra discussing this.
We have 20GB of RAM and 2 Procs and 128GB of diskspace free
28 - /home/updatesd/svn/spamassassin/build/mkupdates/run_nightly is
supposed to do an svn update and reexec but it doesn't work on the first
run properly. Requires 2 runs as best I can tell.
29 - buildbot cron job was:1 1 * * 1
/export/home/automc/svn/spamassassin/build/automc/buildbot.cron.weekly
get buildbot running - Pending JM to see if it's really used anymore - I
don't know if the system was working before the crash and I found a lot
of weirdness in following the chain of files/documents.
31 - I believe I have the sought rule generation system working again
and the rules appear to be getting generated on talon1. However, reach
out to Justin to see if his system on your yerp server still taking and
publishing the channel from there? - EMAILED JUSTIN
-- DONE --
N/A - 1 - Remove keys off devel
DONE- 2 – Figure out opie
Once you are in, you need to do ortpasswd to identify the orthrus key.
In case you miss it first time, I can reset the key.
Once you have successfully done ortpasswd, then try a sudo.
NOTE: This is my challenge key from the first time otp-md5 500 sp9546
http://code.google.com/p/orthrus/wiki/ortcalc
Orthrus provides the ortcalc program for calculating a response.
A typical challenge would look like this:
# ssh myhost.example.com
otp-md5 488 my3414 ext
Password:
In another terminal run:
# ortcalc 488 my3414
Password: ***ENTER YOUR PASSWORD HERE***
Reply: FOND REST PAL BILK ORE BARR
Then copy the reply into the password prompt from ssh.
DONE – NO - 2a – Is there a Windows ortcalc?
DONE - 2b - Learn more about s/key:
http://www.ece.northwestern.edu/CSEL/skey/skey_eecs.html
DONE - 2c – read http://apache.org/dev/freebsd-jails
I used my ldap password
Use OTPDroid on a droid, opiekey on minotaur or ortcalc installed on
talonator. Select md5, cut and paste the Challenge into opiekey, enter
passphrase and cut and paste the password into
spamassassin-vm.apache.org box
DONE - 3 – figure out the old ip
The old IP is free to use, you can add it as an alias to eth0, but do
not change the primary ip please.
DONE - 4 - Please commit all OS config changes to
https://svn.apache.org/repos/infra/infrastructure/trunk/machines/vms/spamassassin-vm.apache.org/
<https://svn.apache.org/repos/infra/infrastructure/trunk/machines/vms/spamassassin-vm.apache.org/>
or send the as patches, so we can commit them. You can also commit appl.
changes in the same place.
DONE - 4a – Ran out of entropy cat /proc/sys/kernel/random/entropy_avail
trying to setup dnssec – Install rngd
DONE - 5 – Note all Commands run on box that change things not covered
in commits
yum install man
yum install lrzsz
yum install bind caching-nameserver
yum install vim
yum install bind-utils
chkconfig named on
yum install rng-tools
chkconfig rngd on
/etc/rc.d/init.d/rngd start
yum install mlocate
yum install screen
yum install jwhois
yum install ntpdate
yum install ntpd
chkconfig ntpd on
/etc/rc.d/init.d/ntpd start
yum install openssh-clients
yum install rsync
yum install mutt
yum install httpd
yum install telnet
DONE - 6 – get DNS working
svn checkout https://svn.apache.org/repos/asf/spamassassin/dns/
/etc/spamassassin-dns
Open ports 53 for udp and tcp
DONE - 6a – Finish README in /etc/spamassassin-dns to reflect new system
and commit
DONE - FIXED - 6b – updatesd isn’t an account and is a static checkout
of trunk as of 4/25 so we can get the tick_serial working
DONE - 6c – Get zones from mirrors – Email Brian Behlendorf and
sonic.net ([Sonic #2659742])
DONE - 6d – Get updates., sought.updates., dnsbltest. and sa-testzones
working
DONE - 6e – start dns or renable transfers ONLY when DNS is fixed
DONE - 6f – commit /etc/spamassassin-dns
N/A - SEE INFORMATION ABOUT CIRCONUS - 7 – Add to nagios with new ip?
DONE - 8 – Remove id_dsa.* on .ssh on hyperion
DONE - 9 – Copy files from hyperion to spamassassin-vm
scp -prC /mnt/zonestorage/spamassassin/
kmcgr...@spamassassin-vm.apache.org
<mailto:kmcgr...@spamassassin-vm.apache.org>:/home/kmcgrail/zonestorage-restore
switch to rsync because of loops on copying files
rsync -e ssh /mnt/zonestorage/spamassassin/
kmcgr...@spamassassin-vm.apache.org
<mailto:kmcgr...@spamassassin-vm.apache.org>:/home/kmcgrail/zonestorage-restore/
--verbose --recursive --relative --links --perms --owner --group --times
--numeric-ids --stats --progress
DONE - ALL GOOD - 10 – Make sure notifies/zone pulls are working -
EMAILED SONIC.NET and HYPERREAL 5/1 to confirm
DONE - ALL GOOD - 11 – Compare a copy of the zone from hyperreal with a
copy from now. - EMAILED FOR COPY
DONE - 13a - make sure ruleqa website works
DONE - restarted /etc/init.d/freqsd daemon but not quite per
https://wiki.apache.org/spamassassin/InfraNotes
DONE - DNS resolves to where? Appears to resolve to zones2
DONE - - where does the data for /export/home/ruleqadb come from? I
think from freqsd - no from freqsd after rsync
DONE - Remove duplicate /etc/shadow line automc:*LK*::::::: and left
automc:NP::::::: which should get rid of ! bad user (automc) Wed Jun 25
02:40:00 2014 in cron log in /var/cron/log
DONE - Check tomorrow and should be good.
DONE - FOUND SYNC SCRIPT - freqsd apparently looks for
/export/home/corpus-rsync/corpus. that dir is stale but contains
weekly-versions.txt and nightly-versions.txt
DONE - 13a2 - Check cron output on zones2
NOTE: Fixed some of cron output from zones2 by adding
/etc/ssh/ssh_keys/<user>.pub from the authorized_keys file and adding
user to sshusers group so that the automated accounts could use ssh.
- bbmass ssh fixed
- bbmass cron, removed --rsync-path=/opt/sfw/bin/rsync since it's not at
that path on sa-vm box
DONE - 13c - Need to document how the ssh works on sa-vm box in case
other accounts are needed to do this on the InfraNotes or similar page
DONE - 14 – setup logrotate for rsyncd.log
DONE - 15 – ASF Infra is phasing out nagios replaced by Circonus. Email
j...@apache.org <mailto:j...@apache.org> re: changes if not handled by 5/8.
DONE - 16 – Look at rsyncd.log file for more things to setup like:
/export/home/bbmass/uploadedcorpora/jhardin_fraud failed: No such file
or directory (2)
2014/05/02 02:00:04 [30700] connect from talon1.pccc.com (38.124.232.50)
2014/05/02 02:00:04 [30700] rsync: chroot
/home/bbmass/uploadedcorpora/guenther_fraud failed: No such file or
directory (2)
DONE - 17 – make test with network tests to confirm DNS is accurate
DONE - 18 - Get httpd working
DONE - LOTS OF DATA AND ACCOUNTS REMAIN BUT NONE BELIEVED NEEDED FOR THE
CURRENT FUNCTIONALITY - 19 - Continue looking at
/root/zonestorage-restore and restoring functionality to the box - THIS
IS THE DATA I WAS ABLE TO RECOVER OFF THE OLD ZONES2 BOX IN THIS DIR
DONE - 20 - Get masschecks running
DONE - 21 - Get updates running
DONE - 24 - restart box and make sure services all start ok and that
hostname sticks - fixed httpd
IGNORING - 25 - Permissions on the box are pretty permissive. BBmass
corpora dirs for example are 777.
DONE - NO - 26 - check cron jobs for jm on talon1 to see if any of them
break because of the new ssh fingerprints
DONE - 30 - This was the traps cron that needs to be added on
spamassassin-vm
20 2 * * * rsync -rze ssh --whole-file --size-only --delete
j...@trap-proc.spamassassin.org
<mailto:j...@trap-proc.spamassassin.org>:/home/jm/cor/.
/export/home/bbmass/uploadedcorpora/traps/.
DONE - add this traps account
DONE - fix perms for /export/home/bbmass/uploadedcorpora/traps/
DONE - add cron job
-- NOTES --
2014-05-01
Got access to backups in /mnt/zonestorage on hyperion
Found date way off - Thu May 1 10:28:49 UTC 2014 changed to 1 May
16:39:21 ntpdate[27179]: step time server 192.5.41.40 offset
22169.764205 sec
Discussed on #ASFINFRA and PCTONY suggested I install NTPD
Transferred a copy of all files on hyperion to /root/zonestorage-restore
on spamassassin-vm.
Created /export dir to symlink things like /home so /export/home/… still
worked
Rsync work:
Created corpus-rysnc user
Renamed user and group to rsync leaving home in /home/corpus-rsync
Mv /root/zonestorage-restore/spamassassin/export/home/corpus-rsync/*
/home/corpus-rsync/
Cleaned up files and changed permissions to what I thought was correct
Compressed old filestar jcvf old_files_2014-05-01.tar.bz2
--remove-files old_files_2014-05-01/
Created automc user (useradd -s /bin/sh -c 'Auto Mass-Check Daemon'
automc)
Moved files from /export/home on zonestorage restoration for automc
Created /home/svn-trunk dir and mv
Moved files from /export/home on zonestorage restoration for svn-trunk
Moved svn-trunk to svn-trunk.old
mkdir svn-trunk
chown automc.automc svn-trunk
su -l -s /bin/bash automc
did a cleancheckout
cd /home; svn checkout
https://svn.apache.org/repos/asf/spamassassin/trunk svn-trunk
ln -s /home/svn-trunk/build/automc/etc-rsyncd.conf /etc/rsyncd.conf
yum install xinetd
set /etc/xinetd.d/rsync to disable=no
Set max connections = 100 in rsyncd.conf
Started xinetd
Allow port 873 tcp/udp on iptables
Restart iptables
Disable selinux with /etc/sysconfig/selinux and setenforce 0
Create logrotate.d/rsync file
Apache work:
Installed httpd
Mv’d files from
/root/zonestorage-restore/spamassassin/backup/rsynced/var-www to /var/www
Believe the conf file is in
/home/automc/svn/spamassassin/build/automc/etc-apache-local-conf-httpd.conf
Open port 80 on iptables and restart
Fixed hostname and hosts
chkconfig httpd on
Disabled Selinux
rebooted
2014-05-02
http://rsync.spamassassin.org returns a 403 message so users can't check
masscheck worked
so I can't check whether it actually worked. - chmod 755 /home/corpus-rsync
2014-06-09
Gave up on getting crontabs via Infra
Some Wiki's to update / look at (Not exhaustive):
SpamAssassinAsfZone
InfraNotes - Updated
RsyncConfig
ApacheSpamassassinWiki:SaUpdateBackend
RescoreMassCheck
AutoRuleUpdates - Good info about mass check
Rysnc work for corpora upload
Created bbmass user ( useradd -s /bin/sh -c 'Buildbot Mass-Check
Daemon' bbmass )
Moved files from /export/home on zonestorage restoration for bbmass
Created symlink to mimic previous but unknown usage - ln -s
bbmass/uploadedcorpora
Moved files I thought were junk in bbmass home dir to /home/bbmass/old
2014-06-13
Found the rsync issue: secrets file must be owned by root when running
as root (see strict modes)
2014-06-14
Found that jhardin_fraud dirs were still an issue.
Tried making them part of the same group with no luck. Changed
/home/bbmass/uploadedcorpora to chown rsync.rsync –R
Cron Analysis to review
Appears to be 5 per day
DONE I BELIEVE - #1 - 10:54PM - received cron from Rule Updates Daemon –
updatesd – Looked to take about 25 minutes to run
bash
/export/home/updatesd/svn/mkupdates-with-scores/do-stable-update-with-scores
runs at 2:25 for UTC
requires perl modules for trunk perl is 5.10.1 with Centos 6.5
installed perl-CPAN from yum
and yum install perl-IO-Socket-INET6 perl-DBI perl-HTML-Parser
perl-Net-DNS perl-NetAddr-IP perl-Time-HiRes perl-IO-Zlib
perl-Archive-Tar perl-IO-Socket-SSL perl-Mail-DKIM wget yum install
perl-Encode-Detect perl-libwww-perl perl-YAML perl-Module-Build
cpan Mail::SPF Net::Patricia IO::Socket::IP Geo::IP
Installed razor 2.84 from source
Make test for trunk then showed some issues such as:
Subroutine IO::Socket::INET6::sockaddr_in6 redefined at
/usr/share/perl5/Exporter.pm line 67.
at /usr/share/perl5/IO/Socket/INET6.pm line 21
SOLUTION: removed perl-IO-Socket-INET6 in favor of IO::Socket::IP
t/dnsbl_subtests.t ................ Can't locate Net/DNS/Nameserver.pm
in @INC (@INC contains: t . /tmp/spamassassin-trunk/blib/lib
/tmp/spamassassin-trunk/blib/arch /usr/local/lib64/perl5
/usr/local/share/perl5 /usr/lib64/perl5/vendor_perl
/usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at
t/dnsbl_subtests.t line 29.
BEGIN failed--compilation aborted at t/dnsbl_subtests.t line 29.
t/dnsbl_subtests.t ................ Dubious, test returned 2 (wstat 512,
0x200)
Failed 46/46 subtests
SOLUTION: Installed perl-Net-DNS-Nameserver
DONE – BUG 7052 - TO-DO: Since perl-Net-DNS-Nameserver is required and
CentOS separately installs it, add it as a requirement to the
Makefile.PL with a note about Centos splitting
http://search.cpan.org/~nlnetlabs/Net-DNS-0.77/
<http://search.cpan.org/%7Enlnetlabs/Net-DNS-0.77/>
MOVED TO A TASK- TO-DO: We may need more CPU power...
- Ran the script and got errors*
- Added updatesd to group for named so scripts could access DNS data -
usermod -a -G named updatesd
- Also added updatesd group to /var/www/buildbot.spamassassin.org with
chgrp updatesd updatestage & chmod g+w updatestage/
- Decided to give chown updatesd.updatesd to updatestage and cleaned
up svn issues.
- Ran script and got errors #2*
- Installed at with yum
- Ran script and got more errors - chown -R updatesd.updatesd
/etc/spamassassin-dns/updates.spamassassin.org.d/ to fix and deleted
some files in updatestage that i think were old remnants with write
protection
- Ran script and got atd error. Started daemon and ran again
- Believe we need pgapack – Still not sure that garescorer is working
correctly on the new server – Noted on
https://wiki.apache.org/spamassassin/RunningGa
- yum install compat-gcc* to get g77
- downloaded pgapack-1.1.1.tar.gz and followed readme to install in
/usr/local/pga
- ./configure -arch linux – Instverf showed many errors but trying 1.1
also had many errors as well. Not sure if this indicates an issue.
- Fixed masses-Makefile.patch to include the new pga dir and path
DONE - #2 11:10PM - - received cron from Rule Updates Daemon - At job –
updatesd
- Changed updatesd mail to go to root
- Checked mail and error in atd job /bin/bash:
/export/home/updatesd/svn/spamassassin/build/mkupdates/tick_zone_serial:
/usr/bin/bash: bad interpreter: No such file or directory
- Changed three scripts in
/export/home/updatesd/svn/spamassassin/build/mkupdates to point to
/bin/bash and re-added same command to atd
- Ran script and got lots of errors
- chown updatesd.updatesd -R spamassassin-dns/
- created counter file with number 4 in it. I believe it
resets daily
DONE - TO-DO: Check the counter file in a day or so from 6/15
- fixed the perl code in tick_zone_serial
- Add to atq with
at -q n now + 1min
/export/home/updatesd/svn/spamassassin/build/mkupdates/tick_zone_serial
ctrl+d
- Fixed sudo: sorry, you must have a tty to run sudo by changing
sudoers to comment out #Defaults requiretty
- tried atq again
- Added updatesd ALL = NOPASSWD: /usr/sbin/rndc reload to
sudoers with visudo
- changed sudo to no tty just for updatesd
DONE - #3 4:31AM - received from Rule Updates Daemon - updatesd
bash /home/updatesd/svn/spamassassin/build/mkupdates/run_nightly
DONE - Fix error
/home/updatesd/svn/spamassassin/build/mkupdates/run_nightly: line 31:
/local/perl586/bin/perl: No such file or directory
DONE - Fix permissions: chown updatesd.updatesd -R
/var/www/ruleqa.spamassassin.org/reports
MOVED TO A TASK - TO-DO - the run_nightly script is supposed to do an
update and reexec but it doesn't work on the first run properly.
Requires 2 runs as best I can tell.
DONE - #4 4:50AM - received from Rule Updates Daemon - updatesd
bash /home/updatesd/svn/spamassassin/build/mkupdates/run_part2
- Getting a lint failure on a rule - publish __FROMN_LII add to
force_active and rerunning from script 1 on
- Script 1 from cron now takes real 14m35.029s to run.
- MOVED TO ANOTHER NOTE- TO-DO - Looking at some logs prior to the
crash, the run_part2 might not have always run because of lints and metas.
DONE - #5 4:53AM - received from release
4/19 - Saturday morning ran weekly
/export/home/svn-trunk/backend/nitemc/extract_to_rsync_dir weekly
/home/corpus-rsync/corpus/weekly-versions.txt >> $HOME/extract.log
or
/export/home/svn-trunk/backend/nitemc/extract_to_rsync_dir nightly
/home/corpus-rsync/corpus/nightly-versions.txt >> $HOME/extract.log
- Changed perms to chown release.release -R release for home dir. This
might break previous permission fixes.
- need the nightly-versions.txt and weekly-versions.txt - restoring from tar
To-do
DONE - Restore updatesd from backups / updatesd:x:114:3002:Rule Updates
Daemon:/export/home/updatesd:/bin/sh / updatesd:kFVKZd/t.lQXE:14448::::::
DONE - Fix Release from backups /
release:x:106:1::/export/home/release:/bin/bash / np set
IGNORING - Many users are members of Group 1. not sure if that's important
DONE - Add kmcgr...@pccc.com <mailto:kmcgr...@pccc.com> to root aliases
DONE - Installed openssl.x86_64 0:1.0.1e-16.el6_5.14
DONE - Installed patch from yum
DONE - Look at
DONE - ./backup/rsynced/etc/
DONE - ./backup/rsynced/export-home-nitemc/my_crontab
DONE - ./backup/rsynced/export-home-automc/my_crontab
N/A - SHOULD BE TAGGED TO REVISION - Remove old logs from #1 in cron
ON ANOTHER TICKET - Make sure everyone in corpora/masscheck is on ruleqa
list
DONE - Cron Items to look at
IGNORING WITH NTPD - Time check from nitemc account
10 6 * * * /usr/sbin/ntpdate -q www.apache.org
<http://www.apache.org> 2>/dev/null | /usr/xpg4/bin/grep -q 'offset
-*[1-9][0-9]' && ( date; /usr/sbin/ntpdate -vq www.apache.org
<http://www.apache.org> 2>/dev/null ) | /usr/bin/mailx -s "WARN:
SpamAssassin ruleqa: spamassassin.zones.apache.org clock drift
excessive" d...@spamassassin.apache.org
<mailto:d...@spamassassin.apache.org> jhar...@impsec.org
<mailto:jhar...@impsec.org>
DONE - Change to use tock and notify root as well
DONE - fix on zones2
10 6 * * * /usr/sbin/ntpdate -q tock.usno.navy.mil 2>/dev/null |
/usr/xpg4/bin/grep -q 'offset -*[1-9][0-9]' && ( date; /usr/sbin/ntpdate
-vq tock.usno.navy.mil 2>/dev/null ) | /usr/bin/mailx -s "WARN:
SpamAssassin ruleqa: spamassassin2.zones.apache.org clock drift
excessive" root d...@spamassassin.apache.org
<mailto:d...@spamassassin.apache.org> jhar...@impsec.org
<mailto:jhar...@impsec.org>
DONE - Open ticket about time drift on SA2
DONE - root
# rsync of good stuff to spamassassin2
4 0,2,4,6,8,10,12,14,16,18,20,22 * * *
/export/home/svn-trunk/build/automc/rsync_to_spamassassin2
#
# backups
4 3 * * * /export/home/svn-trunk/backend/root/backup/backup_zone # &&
/export/home/svn-trunk/backend/root/backup/rsync_spamassassin_to_spamassassin2
DONE - TO-DO: Get the ssh keys working so backups work - - We appear to
have lost the private key that was on spamassassin.zones.apache.org so I
added the key from root@spamassassin2 to root .ssh on spamassassin-vm
and changed authorized_keys to include that public key. This will need
to be repeated on any other servers that zones used to automatically
login to.
MOVED TO A TASK ABOVE- buildbot
1 1 * * 1
/export/home/automc/svn/spamassassin/build/automc/buildbot.cron.weekly
MOVED TO A TASK ABOVE TO-DO: get buildbot running
DONE - release
# Remember, we're in UTC here ...
50 8 * * 0-5 /export/home/svn-trunk/backend/nitemc/corpora_runs >>
/home/corpus-rsync/corpus/nightly-versions.txt
53 8 * * 0-5 /export/home/svn-trunk/backend/nitemc/extract_to_rsync_dir
nightly /home/corpus-rsync/corpus/nightly-versions.txt >> $HOME/extract.log
50 8 * * 6 /export/home/svn-trunk/backend/nitemc/corpora_runs >>
/home/corpus-rsync/corpus/weekly-versions.txt
53 8 * * 6 /export/home/svn-trunk/backend/nitemc/extract_to_rsync_dir
weekly /home/corpus-rsync/corpus/weekly-versions.txt >> $HOME/extract.log
0 0 1 * * mv $HOME/extract.log $HOME/extract.log.1
- chown release.updatesd weekly-versions.txt nightly-versions.txt
- Fix corpora runs
- add to crontab
DONE - automc
automc =====
40 0,1,2 * * *
/export/home/automc/svn/spamassassin/masses/rule-qa/automc/RUNME.before
DONE - Make more account email go to root
updatesd: root
automc: root
bbmass: root
release: root
rsync: root
DONE - updatesd
updatesd =====
30 8 * * * bash /home/updatesd/svn/spamassassin/build/mkupdates/run_nightly
50 8 * * * bash /home/updatesd/svn/spamassassin/build/mkupdates/run_part2
25 2 * * * bash
/export/home/updatesd/svn/mkupdates-with-scores/do-stable-update-with-scores
# dos: testing a script to trigger stable rule updates as soon as we've
got mass-check results in
# 10,25,40,55 0-4,13-23 * * * bash
/export/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores-expedite
# 6 6 * * * rm -f
/export/home/updatesd/state/mkupdate-with-scores-expedite.state
MOVED TO TASK ABOVE - traps =====
20 2 * * * rsync -rze ssh --whole-file --size-only --delete
j...@trap-proc.spamassassin.org
<mailto:j...@trap-proc.spamassassin.org>:/home/jm/cor/.
/export/home/bbmass/uploadedcorpora/traps/.
DONE - add this traps account
MOVED TO tasks above - fix perms for
/export/home/bbmass/uploadedcorpora/traps/
MOVED TO TASKS ABOVE - add cron job
TO-DO:
IGNORING - NOTE: From reviewing the old password file, buildbot, bbmass
and automc are in a group id 3000.
Made bbmass belong group to 4471 which was automc’s group. Buildbot
doesn’t exist as a user at this point (6/14).
DONE - Task 52336 #16 is solved when corpora uploads work & reply to
mailing list letting him know that part is restored for john hardin
ON ANOTHER TASK - 49276 - Make sure everyone active with RuleQA is on
mailing list including corpora upload
DONE - ADDED ANYWAY - Update
http://wiki.apache.org/spamassassin/UploadedCorpora if chmod rsync.rsync
is required.
DONE - Setup jhardin’s account
MOVED TO ANOTHER NOTE- Install Crashplan
* Errors 2014-06-15
Error set #2
/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores: line 364:
at: command not found
+ cd
+ at -q n now + 16min
/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores: line 362:
at: command not found
+ echo
/export/home/updatesd/svn/spamassassin/build/mkupdates/tick_zone_serial
Exit Status 127 is not zero for mkupdate-with-scores
Error Set #1
+ STATUS=0
+ set -e
+ cd ..
+ rm -rf release_3.3.3 /tmp/sa-mkupdate-6153/release_3.3.3
+ return 0
+ update_dns_record 3.3.3 1588859
+ SA_VERSION=3.3.3
+ UPDATE_REVISION=1588859
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
++ perl -pe 's/^(\d+)\.(\d+)\.(\d+)$/$3.$2.$1/'
++ echo 3.3.3
+ RVERS=3.3.3
+ DNS_RECORD='3.3.3 TXT "1588859"'
+ echo 'DNS Record: 3.3.3 TXT "1588859"'
DNS Record: 3.3.3 TXT "1588859"
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.3
+ mkdir /tmp/sa-mkupdate-6153/dns-backup
mkdir: cannot create directory `/tmp/sa-mkupdate-6153/dns-backup': File
exists
+ set +e
+ cp /var/named/updates.spamassassin.org.d/3.3.3
/tmp/sa-mkupdate-6153/dns-backup/.
cp: cannot stat `/var/named/updates.spamassassin.org.d/3.3.3':
Permission denied
+ set -e
+ echo 3.3.3 TXT '"1588859"'
/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores: line 125:
/var/named/updates.spamassassin.org.d/3.3.3.mkupdate-with-scores.new:
Permission denied
+ mv
/var/named/updates.spamassassin.org.d/3.3.3.mkupdate-with-scores.new
/var/named/updates.spamassassin.org.d/3.3.3
mv: accessing `/var/named/updates.spamassassin.org.d/3.3.3': Permission
denied
+ return 0
+ UPDATED_VERSIONS=4
+ echo 'VERSIONS UPDATE PASSED ON: 4'
VERSIONS UPDATE PASSED ON: 4
+ '[' 4 -gt 0 ']'
+ EXIT=0
+ copy_update_paranoid /tmp/sa-mkupdate-6153/1588859.tar.gz
/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
+ SRC=/tmp/sa-mkupdate-6153/1588859.tar.gz
+ DST=/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
+ set +e
+ cp /tmp/sa-mkupdate-6153/1588859.tar.gz
/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
cp: cannot create regular file
`/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz':
Permission denied
+ diff -u /tmp/sa-mkupdate-6153/1588859.tar.gz
/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
diff: /var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz: No
such file or directory
+ '[' 2 -ne 0 ']'
+ set -e
+ return 1
+ EXIT=5
+ '[' 5 -gt 0 ']'
+ (( I=0 ))
+ (( I<=3 ))
+ revert_dns_record 3.3.0
+ SA_VERSION=3.3.0
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.0
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.0
/var/named/updates.spamassassin.org.d/3.3.0
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.0': Permission
denied
+ set -e
+ (( I++ ))
+ (( I<=3 ))
+ revert_dns_record 3.3.1
+ SA_VERSION=3.3.1
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.1
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.1
/var/named/updates.spamassassin.org.d/3.3.1
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.1': Permission
denied
+ set -e
+ (( I++ ))
+ (( I<=3 ))
+ revert_dns_record 3.3.2
+ SA_VERSION=3.3.2
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.2
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.2
/var/named/updates.spamassassin.org.d/3.3.2
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.2': Permission
denied
+ set -e
+ (( I++ ))
+ (( I<=3 ))
+ revert_dns_record 3.3.3
+ SA_VERSION=3.3.3
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.3
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.3
/var/named/updates.spamassassin.org.d/3.3.3
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.3': Permission
denied
+ set -e
+ (( I++ ))
+ (( I<=3 ))
+ exit 5
Exit Status 5 is not zero for mkupdate-with-scores
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
