Re: Ethics Guidelines was Fwd: Re: > The machines that run sa-update stuff are offline at the moment.

[Kevin A. McGrail]

On 6/21/2017 3:26 AM, Frank Urban wrote:
that's a lot of information now. I'm not sure what kind of job you are 
planning for me. You wrote "Admin" and in a later email "Committer". I 
have no idea what will be the job of a Committer.
The documents from Jochen seemed to be created for someone who is 
looking for a job for the CIA or FBI :)
I think I can understand enough from the LISA guidelines to agree to 
them.
I'm still working as Admin and can access a lot of high secure data. 
So I know what is allowed and what's not.


Hi Frank, yes, Joey did a very good job :-)  Perhaps too good.  But your 
answer suffices.


re: Admin v Committer

SysAdmins for SpamAssassin use a lot of SVN to store data for the 
machine administration.  Because of this you will have committer access 
for sysadmin purposes.  Does that make sense?


Regards,
KAM

Re: Things just keep happening

[Kevin A. McGrail]

I consider you all part of the ASF family so I am glad things are looking 
better.  Never too much info especially if it helps you feel better to know we 
understand. 

The project will still be here so no sweat on our part and we welcome any help 
you can give whenever that is.

Anything you need?
Regards,
KAM

On June 21, 2017 9:36:02 AM EDT, Bryan Vest  wrote:
>Just as I was starting to pick up speed here I ended up in the hospital
>for
>4 days. I went in because of severe chest pain, my heart rate had
>dropped
>to 35bpm. The local hospital could not figure out what was going on and
>could not get it above 40bpm. After 12 hours of still being in pain
>they
>threw an external pacemaker on me and transfered me by ambulance to the
>nearest hospital that specializes in heart problems.
>
>The new hospital ran more tests than any human needs and said "You have
>all
>of the symptoms of a heart attack but your blood work is not showing
>the
>enzymes associated with a heart attack."
>
>They tried atropine, that worked for a few minutes getting my heart up
>to
>52bpm then it dropped back to the 35 - 40 bpm range.
>
>They changed tactics and said, lets try getting you out of pain and see
>if
>that helps. So after being hooked up to multiple machines to monitor
>every
>part of me they gave me a dose of morphine through IV. As the pain
>faded my
>heart started to stabilize but was still not where it should have been.
>The
>next day they gave me a a shot of Valium and things started to come
>back to
>normal.
>
>Their final diagnosis was that I had been in a severe panic attack for
>3+
>days.
>
>Maybe TMI but just wanted to let everyone know why I went silent all of
>a
>sudden.
>
>--Bryan

Ethics Guidelines was Fwd: Re: > The machines that run sa-update stuff are offline at the moment.

[Kevin A. McGrail]

Frank, my friend Joey who is also German wrote back.  He has been on 
holiday and apologizes for the delay!


He would like to help make sure you are good to go with the project.  
I've cc'd him and the other sysadmins on the project.


Because people donate us their mail corpora of both ham & spam, there is 
a level of ethics that the LISA/Sage guidelines provide that we need to 
move forward.


Since you weren't sure you understood everything in the LISA/Sage ethics 
guidelines because it was in English, I've asked my friend, Joey to help.


He will help you make sure you can agree to the LISA/Sage ethics OR any 
substantially similar sysadmin ethics guidelines that he approves of in 
German.


He wrote some notes below too.


Regards,

KAM


I looked up some Ethics Guides that I adhere to:
ISACA
https://www.isaca.org/Certification/Code-of-Professional-Ethics/Pages/default.aspx
https://www.isaca.org/About-ISACA/History/Deutsch/Documents/ISACA-Code-of-Ethics-German.pdf

and also ISC2' Code of Ethics:
https://www.isc2.org/ethics/default.aspx


A more detailed version if the general principles of ISACA and ISC is 
the following:


As I am also a registered Auditor for the German Federal Government I 
also adhere to the following Ethics Code:
PDF: 
https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Zertifizierung/Prog-Personen.pdf?__blob=publicationFile=6
Page 8 / 89 Chapter "2.1.1 Die persönlichen Eigenschaften eines 
Auditteamleiters"


If you have him counter-sign the following german PDF: 
Prog-Personen-Chapter_2.1.1.-german
then you have as much dude diligence done as I can think of before 
entering into a contractual agreement.


NOTE: if you give Joey your phone number, he can phone him about the 
attached PDF.


p.s.
If he has by chance an active ISACA or ISC2-title (you get those as an 
online PDF) he has also accepted the above Codes of Ethics.

Slack Conversations

[Kevin A. McGrail]

I asked Chris if I could forward this as it will bring a thought process as you 
guys learn about managing things under the asf.
Regards,
KAM



I have an action item from the Board to follow up with something mentioned in
your prior report. It seems in that report there is mention of Slack 
conversations
in the project. While these are fine, project decisions are made on the mailing 
list, 
in a timely fashion. Documenting decisions *after the fact* can lead to 
community erosion and
reduce overall participation by those that feel marginalized if they are not 
directly 
part of the Slack conversation.

Please try to balance that, and to ensure that everyone can participate in the 
project.

Thanks,
Chris Mattmann
(on behalf of the ASF Board)

Re: Fwd: Re: Errors since upgrading to 3.4.1: "meta test ... with a zero score"

[Kevin A. McGrail]

Thought.  Get the update nums from previous turn back on and copy those files 
to a higher number and update dns.  That will revert back to last known good. 
Regards,
KAM

On June 15, 2017 9:24:37 PM EDT, Dave Jones  wrote:
>Ugg!  I asked for some help QA'ing the rules for over a week but got 
>zero response then.  I wonder if what was in SVN wasn't what was really
>
>running on the old masscheck box.  I used what was in SVN.  I guess I 
>will dig through the old VM backup to see if I can find the difference 
>related to this issue.
>
>Dave
>
> Forwarded Message 
>Subject: Re: Errors since upgrading to 3.4.1: "meta test ... with a
>zero 
>score"
>Date: Thu, 15 Jun 2017 18:00:28 -0700
>From: John Hardin 
>To: us...@spamassassin.apache.org
>
>On Thu, 15 Jun 2017, Gerald Turner wrote:
>
>>  spamd[32137]: rules: meta test FREEMAIL_FORGED_FROMDOMAIN has
>dependency 'HEADER_FROM_DIFFERENT_DOMAINS' with a zero score
>>  spamd[31552]: rules: meta test __FORM_FRAUD_3 has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __MONEY_FRAUD_3 has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __FORM_FRAUD_5 has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __ADVANCE_FEE_4_NEW has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __MONEY_FRAUD_8 has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __ADVANCE_FEE_2_NEW has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __MONEY_FRAUD_5 has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __ADVANCE_FEE_3_NEW has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __ADVANCE_FEE_5_NEW has dependency
>'LOTTO_AGENT' with a zero score
>>  spamd[31552]: rules: meta test __FORM_FRAUD has dependency
>'LOTTO_AGENT' with a zero score
>
>>  - Is there a bug with the project's sa-update channel / auto-
>>mass-check setup?
>
>That's what it sounds like to me - it should not be omitting or zeroing
>
>the scores of rules that participate in metas.
>
>Something is odd. This didn't come up on the old masscheck host, but
>the 
>score generation code should not have changed since then...
>
>It looks like it's not setting both the net and non-net scores for a
>few
>rules:
>
>   score FROM_IN_TO_AND_SUBJ1.099 0.000 1.099 0.000
>   score HEADER_FROM_DIFFERENT_DOMAINS  0.001 0.000 0.001 0.000
>   score HK_SCAM_N8 2.506 0.000 2.506 0.000
>   score LOTTO_AGENT2.609 0.000 2.609 0.000
>
>The non-network-enabled scores should only be zero for rules marked as 
>being network-dependent rules, and *all* rules should have a nonzero 
>network-enabled score (which appears to be the problem here).
>
>Something else odd is going on in the score generation: some 
>well-performing rules (notably URI_WP_HACKED) are now getting scored at
>
>1 point. There are only 56 rules listed in 72_scores.cf (the output
>from 
>the masscheck score generator), the rest would be defaulting to 1
>point.
>
>
>-- 
>  John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
>  jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
>  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
>---
>   If you ask amateurs to act as front-line security personnel,
>   you shouldn't be surprised when you get amateur security.
> -- Bruce Schneier
>---
>  3 days until SWMBO's Birthday

Re: Fwd: New CrashPlan Backup Alerts

[Kevin A. McGrail]

On 6/15/2017 1:30 PM, Dave Jones wrote:
It is Java...  We could setup supervisord to make sure it's running. 
Joking aside, I have this on a lot of boxes.  We changed RAM & CPUs.  
I'll keep an eye on it for now.

Re: Think I locked myself out of the SVN

[Kevin A. McGrail]

On 6/14/2017 11:01 AM, Bryan Vest wrote:

hipchat was great, I had actually locked out my IP. Chris Thistlewaite
worked with me to get it unblocked and tested.

--Bryan


Perfect!  I try not to use hipchat unless it's time-sensitive, FYI.

Re: Main Developer

[Kevin A. McGrail]

On 6/14/2017 10:19 AM, Bryan Vest wrote:

There are deeper details, this is just the summary. Just throwing this in
the mix for my agreement SA is no doubt very versatile.


:-) Agreed.  It's an amazing tool.  I use it for things other than email 
as well :-)

Re: Main Developer

[Kevin A. McGrail]

On 6/14/2017 8:40 AM, Bryan Vest wrote:

Yes I keep up with rspamd, read into what it can do but have not tested it
yet. I have read mentions of SA V4 in the different lists. Where does that
stand? Are there any dev notes for changes/additions in V4?


v4 the big difference is switching to more of a UTF-8 core which will 
then move 3.4.X to maintenance only.


3.4.2 is in progress and doing very well but snowballed in the fact that 
I keep finding more things to fix/patch to be "done". I'm drawing a line 
in the sand today as it will never be "done".



Regards,
KAM

Re: bugzilla canconfirm Can confirm a bug.

[Kevin A. McGrail]

On 6/12/2017 3:14 PM, Bryan Vest wrote:

I already have the keys to start the engine, might as well give me the keys
to the that castle too.:)


Done!

Re: Main Developer

[Kevin A. McGrail]

Some comments in-line below:

On 6/14/2017 9:47 AM, Dave Jones wrote:
There are soo many bugs already open.  I looked around BZ some 
last week and it seems like there is no real recent movement on 
anything.  I would like to create a BZ to get a DMARC plugin started 
but on the users mailing list a few weeks back it didn't seem to get 
any traction or enough interest.  It seems like so many people are 
using SA in many different ways that the momentum to move things 
forward is fragmented. The great thing about SA is that it's very 
flexible but that is also a negative thing too.
I would appreciate it if you would still open a bug.  I am trying to 
make that a central place for ideas.


Also, you might try asking someone to write something or post a draft 
patch, etc.  A little bit of movement can become a snowball.
Maybe there are some out there that took some of my techniques and are 
trying them out but it seems that everyone is kinda set in their ways. 
Mail filtering is changing with SPF, DKIM, DMARC, ARC, etc.  RBLs are 
still very important but SA currently doesn't use enough of them by 
default.  I understand they have to meet some requirements before they 
can be included in SA by default which is why I am pushing this in BZ.
Remember that at least my POV is that SA is a framework so the goal for 
me is to support RBLs in general.  But I'm very happy to support you in 
this endeavor.


At Apache, it's code over community.  Think about how you can get others 
to help.
There are some brilliant people on the users list that do mail 
filtering as their business but they don't want to give away their 
"secret sauce" and I get that.  But we should be able to elevate the 
entire SA community by improving the default SA configurations and 
rules.  It basically took me a few years of using SA part time to 
figure everything out which is too high of a barrier of entry for most 
sysadmins who do mail filtering as one of many roles.


Agreed.  There is also an issue where email has become a commodity that 
Office 365, or Google or GoDaddy provide for a small outlay of cash.  
The number of sysadmins is becoming rarer each day.


Regards,

KAM

Re: Fwd: Re: [Bug 7432] Update cron jobs on sa-vm1 to run at UTC and ignore daylight savings change

[Kevin A. McGrail]

My son is graduating tomorrow, turning 18 on Friday and finalizing his 
Eagle Scout at the last minute. My life is in a little tornado at the 
moment too!


BTW, about 3 years ago we had a multiday power outage for a large 
segment of the area where I live but my office had power because some 
acronyms that shall not be named made all of that areas power more 
resilient.


We invited kids to play and adults to hang out, etc.  Also air 
conditioning... in DC... in the Summer...  So yeah, full house  
Internet and I used to lead a FIRST Lego League so I had LOTS of legos 
at the office.


After hanging out there with lights, internet, ac, etc., for about 8 
hours, my wife just wanted to go home.  First thing she did when we got 
home was tell me the internet wasn't working ;-)


On 6/14/2017 7:17 AM, Dave Jones wrote:
I had a power outage yesterday evening for a few hours so I didn't get 
to this yet.  I will update and close the bug when it's done.


P.S. My UPSes held up for about 45 minutes so I could finish some day 
job work.  After that it's very interesting how much we take for 
granted when the power is on.  You walk into any dark room and 
automatically flip the light switch.  No microwave to heat up some 
food.  I don't even have a battery powered radio to listen to the 
Texas Rangers game.  :)


Dave


 Forwarded Message 
Subject: Re: Fwd: Re: [Bug 7432] Update cron jobs on sa-vm1 to run at 
UTC and ignore daylight savings change

Date: Tue, 13 Jun 2017 13:31:02 -0500
From: Dave Jones <da...@apache.org>
To: sysadmins@spamassassin.apache.org

Thanks, In a few hours, I will change the TZ and update the 
/etc/cron.d/automc with the real hours in UTC.  This will be much better!


Dave

On 06/13/2017 01:11 PM, Kevin A. McGrail wrote:

FYI



 Forwarded Message 
Subject: Re: [Bug 7432] Update cron jobs on sa-vm1 to run at UTC 
and ignore daylight savings change

Date: Tue, 13 Jun 2017 10:33:19 -0700
From: Chris Lambertus <c...@apache.org>
To:     Kevin A. McGrail <kevin.mcgr...@mcgrail.com>
CC: Apache Infrastructure <infrastruct...@apache.org>



You can change it. Puppet is supposed to update it to UTC, but I have 
seen a few circumstances where this isn't happening on 16.04.



On Jun 11, 2017, at 6:12 PM, Kevin A. McGrail 
<kevin.mcgr...@mcgrail.com <mailto:kevin.mcgr...@mcgrail.com>> wrote:


Question, on sa-vm1, can we change the machine to UTC or do we need 
a Jira ticket?





 Forwarded Message 
Subject: [Bug 7432] Update cron jobs on sa-vm1 to run at UTC and 
ignore daylight savings change

Date: Sun, 11 Jun 2017 16:45:36 +
From: bugzilla-dae...@issues.apache.org
Reply-To: sysadmins@spamassassin.apache.org
To: sysadmins@spamassassin.apache.org



https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7432

Dave Jones<da...@apache.org>  changed:

What|Removed |Added
 


  CC| |da...@apache.org

--- Comment #1 from Dave Jones<da...@apache.org>  ---
It would be nice to run the server at UTC to solve this problem and 
not have to
keep calculating file timestamp offsets when we run 'ls -l'. We 
would need to
run this past the INFRA team.  I think they have Puppet managing the 
timezone
file but I am not sure.  We may want to try to change the system 
timezone to
UTC and see if Puppet puts it back to "America/Los_Angeles" then 
open a Jira

issue to ask the INFRA team about running the server at UTC.

--
You are receiving this mail because:
You are the assignee for the bug.

Re: Main Developer

[Kevin A. McGrail]

On 6/14/2017 8:04 AM, Dave Jones wrote:

That seems correct from what I have seen.

Speaking of development.  rspamd is starting to really take off:

https://rspamd.com/

I hope we can keep SA out there as the leader that everyone compares 
themselves to:


https://rspamd.com/comparison.html

There are some meaningless red X's under SA like greylisting and rate 
limiting that are done outside of SA but the DMARC and ARC support 
seem to be way ahead.  SA basically has no built-in support for DMARC 
or the new extension of ARC. 


That would be something to mention on dev@ or users@.  I would also 
suggest bugzilla bugs being opened to do this with as much information 
as you can muster.

Re: Think I locked myself out of the SVN

[Kevin A. McGrail]

On 6/14/2017 9:21 AM, Bryan Vest wrote:

I received an email from Sidney Markowitz that my SVN access had bren setup
and to test a commit by editing the CREDITS file.

I think I locked myself out using the wrong password. Before I open a
ticket with INFRA I wanted to make sure that is the correct route to go.

I used SVN for maybe 2 months before we switched to using git so more to
learn.

And you can use hipchat https://www.apache.org/dev/infra-contact to see 
if they can help with unlocking.  Since this is your first work as a 
committer, you might want more informality and conversation, etc.

Re: bugzilla canconfirm Can confirm a bug.

[Kevin A. McGrail]

On 6/12/2017 2:50 PM, Dave Jones wrote:
I am fine with leaving the existing perms to be all or nothing.  No 
need to make more work if it's not necessary.  My comment was more 
toward the "take" link/option should be available to all accounts by 
default.  I guess BZ wasn't setup that way. 


You might just know more about BZ than others on the project. Feel free 
to discuss on dev@team and if you get lazy consensus, then JDI.


For example, what I mean is:


"Hey guys, I think it would be helpful to make this change in bugzilla 
to do X so we can do Y.  Let me know if you disagree or have comments 
otherwise I'll make the change "



This would come under the concept of lazy consensus.

Best,

KAM

Re: Main Developer

[Kevin A. McGrail]

On 6/14/2017 7:58 AM, Bryan Vest wrote:

This may seem like a question I should know the answer to and I think I do
but just want to verify. Who is the main developer on SA? I know overall
development is a crew of volunteer's and would assume Kevin has the final
say, just want to make sure.

--Bryan

Great question and NOT how it works at all sorry to say.  But it's ok 
because I knowingly created sysadmins out of people not committing to 
the project so it makes sense you wouldn't know this.  And it gives me 
the opportunity to introduce you guys a bit to something we call the 
Apache Way.


There is no main developer.  I just happen to have been around a long 
time and I'm working to make sure the community is healthy.


NOTE: I am, however, the volunteer for the release manager so you'll see 
me making decisions solely to build consensus and get it done.



Plus we work on a system of meritocracy where leadership is by merit 
earned.  Hopefully, I've earned a lot of merit and there is a method of 
Just Do It (tm) so when you see something that needs doing, you have the 
keys to the castle and you get it done.  You can see this in certain 
stances on Review Then Commit (RTC) and Commit Then Review (CTR).


We have a project management committee and a VP of the PMC reports to 
the board.  But everything is by consensus.


This type of programming methodology permeates how the entire ASF operates.

Some info about this at Apache: 
https://www.apache.org/foundation/voting.html


https://www.apache.org/foundation/how-it-works.html

This will make more sense over time.  A lot of people are like WTF and 
it takes a few years and then they really embrace it.

Re: Fwd: Re: [Bug 7432] Update cron jobs on sa-vm1 to run at UTC and ignore daylight savings change

[Kevin A. McGrail]

On 6/13/2017 2:31 PM, Dave Jones wrote:
Thanks, In a few hours, I will change the TZ and update the 
/etc/cron.d/automc with the real hours in UTC.  This will be much better! 

ty.

Fwd: Re: [Bug 7432] Update cron jobs on sa-vm1 to run at UTC and ignore daylight savings change

[Kevin A. McGrail]

FYI



 Forwarded Message 
Subject: 	Re: [Bug 7432] Update cron jobs on sa-vm1 to run at UTC and 
ignore daylight savings change

Date:   Tue, 13 Jun 2017 10:33:19 -0700
From:   Chris Lambertus <c...@apache.org>
To: Kevin A. McGrail <kevin.mcgr...@mcgrail.com>
CC: Apache Infrastructure <infrastruct...@apache.org>



You can change it. Puppet is supposed to update it to UTC, but I have 
seen a few circumstances where this isn't happening on 16.04.



On Jun 11, 2017, at 6:12 PM, Kevin A. McGrail 
<kevin.mcgr...@mcgrail.com <mailto:kevin.mcgr...@mcgrail.com>> wrote:


Question, on sa-vm1, can we change the machine to UTC or do we need a 
Jira ticket?





 Forwarded Message 
Subject: 	[Bug 7432] Update cron jobs on sa-vm1 to run at UTC and 
ignore daylight savings change

Date:   Sun, 11 Jun 2017 16:45:36 +
From:   bugzilla-dae...@issues.apache.org
Reply-To:   sysadmins@spamassassin.apache.org
To: sysadmins@spamassassin.apache.org



https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7432

Dave Jones<da...@apache.org>  changed:

What|Removed |Added

  CC||da...@apache.org

--- Comment #1 from Dave Jones<da...@apache.org>  ---
It would be nice to run the server at UTC to solve this problem and not have to
keep calculating file timestamp offsets when we run 'ls -l'.  We would need to
run this past the INFRA team.  I think they have Puppet managing the timezone
file but I am not sure.  We may want to try to change the system timezone to
UTC and see if Puppet puts it back to "America/Los_Angeles" then open a Jira
issue to ask the INFRA team about running the server at UTC.

--
You are receiving this mail because:
You are the assignee for the bug.




signature.asc
Description: PGP signature

Re: Cron <automc@sa-vm1> ~/svn/trunk/build/mkupdates/do-stable-update-with-scores > /dev/null

[Kevin A. McGrail]

I am also working on a patch for sha256 fyi if you see oddities.
Regards,
KAM

On June 13, 2017 10:39:53 AM EDT, Dave Jones  wrote:
>That's an optional module caused by the "make" process.  I did have to 
>install a few perl modules to get the make to work but I didn't worry 
>about the optional modules.
>
>Dave
>
>On 06/13/2017 09:29 AM, Bryan Vest wrote:
>> I noticed this in the output above "Warning: prerequisite
>Digest::SHA1 0
>> not found." That looks like a missing per module. Is it removed on
>purpose?
>> 
>> --Bryan
>> 
>> On Mon, Jun 12, 2017 at 10:26 PM, Cron Daemon
>
>> wrote:
>> 
>>> + echo ''\''Running do-nightly-rescore-example.sh ...'
>>> + echo ''
>>> + /usr/local/spamassassin/automc/svn/masses/rule-update-
>>> score-gen/do-nightly-rescore-example.sh
>>> + svn co -r 1798299
>http://svn.apache.org/repos/asf/spamassassin/trunk
>>> trunk-new-rules-set1
>>> + svn co http://svn.apache.org/repos/asf/spamassassin/tags/
>>> spamassassin_release_3_3_0/rules trunk-new-rules-set1/rules-base
>>> + svn co http://svn.apache.org/repos/asf/spamassassin/trunk/rules
>>> trunk-new-rules-set1/rules-current
>>> + svn up -r 1798299 trunk-new-rules-set1/rulesrc/
>>> + svn up trunk-new-rules-set1/masses/
>>> + svn up trunk-new-rules-set1/build/
>>> + set +x
>>> Warning: prerequisite Digest::SHA1 0 not found.
>>> Month distribution:
>>>
>>> 75621 ( 56%) 75621   0-1 months old
>>> 84834 ( 63%)  9213   1-2 months old
>>> 87253 ( 65%)  2419   2-3 months old
>>> 89525 ( 67%)  2272   3-4 months old
>>> 91661 ( 68%)  2136   4-5 months old
>>> 93647 ( 70%)  1986   5-6 months old
>>> 95812 ( 71%)  2165   6-7 months old
>>> 97652 ( 73%)  1840   7-8 months old
>>> 99585 ( 74%)  1933   8-9 months old
>>>101579 ( 76%)  1994   9-10 months old
>>>103567 ( 77%)  1988  10-11 months old
>>>105373 ( 78%)  1806  11-12 months old
>>>106683 ( 79%)  1310  12-13 months old
>>>108173 ( 81%)  1490  13-14 months old
>>>109598 ( 82%)  1425  14-15 months old
>>>111388 ( 83%)  1790  15-16 months old
>>>113001 ( 84%)  1613  16-17 months old
>>>114258 ( 85%)  1257  17-18 months old
>>>115661 ( 86%)  1403  18-19 months old
>>>117113 ( 87%)  1452  19-20 months old
>>>118512 ( 88%)  1399  20-21 months old
>>>119786 ( 89%)  1274  21-22 months old
>>>121023 ( 90%)  1237  22-23 months old
>>>122273 ( 91%)  1250  23-24 months old
>>>123540 ( 92%)  1267  24-25 months old
>>>124743 ( 93%)  1203  25-26 months old
>>>126044 ( 94%)  1301  26-27 months old
>>>126884 ( 95%)   840  27-28 months old
>>>127011 ( 95%)   127  28-29 months old
>>>127105 ( 95%)94  29-30 months old
>>>127238 ( 95%)   133  30-31 months old
>>>127319 ( 95%)81  31-32 months old
>>>127438 ( 95%)   119  32-33 months old
>>>127519 ( 95%)81  33-34 months old
>>>127597 ( 95%)78  34-35 months old
>>>127756 ( 95%)   159  35-36 months old
>>>127856 ( 95%)   100  36-37 months old
>>>127932 ( 95%)76  37-38 months old
>>>128027 ( 95%)95  38-39 months old
>>>128115 ( 95%)88  39-40 months old
>>>128324 ( 96%)   209  40-41 months old
>>>128418 ( 96%)94  41-42 months old
>>>128675 ( 96%)   257  42-43 months old
>>>128869 ( 96%)   194  43-44 months old
>>>128957 ( 96%)88  44-45 months old
>>>129113 ( 96%)   156  45-46 months old
>>>129259 ( 96%)   146  46-47 months old
>>>129397 ( 96%)   138  47-48 months old
>>>129496 ( 96%)99  48-49 months old
>>>129616 ( 97%)   120  49-50 months old
>>>129670 ( 97%)54  50-51 months old
>>>129729 ( 97%)59  51-52 months old
>>>129818 ( 97%)89  52-53 months old
>>>129884 ( 97%)66  53-54 months old
>>>129928 ( 97%)44  54-55 months old
>>>129949 ( 97%)21  55-56 months old
>>>130114 ( 97%)   165  56-57 months old
>>>130388 ( 97%)   274  57-58 months old
>>>130621 ( 97%)   233  58-59 months old
>>>130961 ( 98%)   340  59-60 months old
>>>131237 ( 98%)   276  60-61 months old
>>>131440 ( 98%)   203  61-62 months old
>>>131496 ( 98%)56  62-63 months old
>>>132107 ( 98%)   611  63-64 months old
>>>132646 ( 99%)   539  64-65 months old
>>>133053 ( 99%)   407  65-66 months old
>>>133361 ( 99%)   308  66-67 months old
>>>133384 ( 99%)23  67-68 months old
>>>133397 ( 99%)13  68-69 months old
>>>133406 ( 99%) 9  69-70 months old
>>>133417 ( 99%)11  70-71 months old
>>>133420 ( 99%) 3  71-72 months old
>>>133427 ( 99%) 7  72-73 months old
>>>133444 ( 99%)17  73-74 

Re: bugzilla canconfirm Can confirm a bug.

[Kevin A. McGrail]

Never really thought about it much.  Most of the project committees just ask 
and get everything.

If you want less than everything I can send you a list and you can tell me 
which you want.
Regards,
KAM

On June 12, 2017 2:28:16 PM EDT, Dave Jones  wrote:
>Really?  I would think that anyone would be allowed to "take" a bug 
>without needing "keys to the castle."
>
>Dave
>
>On 06/12/2017 07:54 AM, Bryan Vest wrote:
>> I only seem to have the bugzilla permission "canconfirm Can confirm a
>bug".
>> Should I have different permissions?
>> 
>> Just making sure everything is like it should be. My login there is
>> bv...@apache.org.
>> 
>> --Bryan
>> 

Re: bugzilla canconfirm Can confirm a bug.

[Kevin A. McGrail]

There is no set level of permissions per se.

Would you like the keys to the castle?

On 6/12/2017 8:54 AM, Bryan Vest wrote:

I only seem to have the bugzilla permission "canconfirm Can confirm a bug".
Should I have different permissions?

Just making sure everything is like it should be. My login there is
bv...@apache.org.

--Bryan

Re: Minor problem with ruleqa

[Kevin A. McGrail]

Just an FYI that I did not get time on this today.

On 6/10/2017 10:38 PM, Kevin A. McGrail wrote:

Ok.  I will put at least two hours on it.when I get home tomorrow at noon.
Regards,
KAM

On June 10, 2017 10:12:54 PM EDT, Dave Jones <da...@apache.org> wrote:

Kevin,
There seems to be a minor problem with the ruleqa CGI processing.  When

I click a highlighted rule like "RCVD_IN_XBL", the next screen has the
"not found" link:

http://ruleqa.spamassassin.org/20170610-r1798299-n/RCVD_IN_XBL/detail

Detailed results for rule RCVD_IN_XBL, from source file (not found).
Source file was last modified on 1970-01-01 00:00:00 UTC.


Since this is perl, I need some help tracking down what is not running
that builds that detail#rulemetadata link.

--
Dave Jones

Re: DNS history in SVN

[Kevin A. McGrail]

On 6/11/2017 12:55 PM, Dave Jones wrote:
I have setup sa-vm1:/usr/local/bin/pushDNStoSVN.sh to run weekly to 
update SVN (dns repo) with the current zone dump. This will give us 
history of all of the DNS changes for the spamassassin.org zone.


We will get an email from the script via this list and from the SVN 
commit itself via the commits list.


If you feel like this script should run daily, I can change the 
/etc/cron.d/svn to '@daily' instead of '@weekly'.  I thought weekly 
would be enough to start since it's really only the sa-update TXT 
records that are changing daily so far. 


Weekly sounds very good to me.  As you saw before it was just done 
anecdotally prior!

Re: checkMasscheckContribs.sh on sa-vm1.apache.org - not enough contributors for SVN 1798299

[Kevin A. McGrail]

Yes, good point.  Need me to check any old logs for the delay?
Regards,
KAM

On June 10, 2017 8:19:50 AM EDT, Dave Jones  wrote:
>I need to update this to wait longer on Saturdays since the net checks 
>are running today.  I will add this logic now to the script.
>
>Dave
>
>On 06/10/2017 07:17 AM, root wrote:
>> Corpus total: 128, Old: 80, Recent: 16, New: 6
>> 
>> SVN tagged rev in weekly_mass_check:  1798299
>> 
>> New masscheck submission listings in the past day:
>> SVN rev (Match) File Name (Date)
>> 1798299 (Yes) - spam-net-grenier.log (Jun 10 09:04)
>> 1798299 (Yes) - spam-net-thendrikx.log (Jun 10 09:25)
>> 1798299 (Yes) - ham-net-grenier.log (Jun 10 09:04)
>> 1798299 (Yes) - spam-net-darxus.log (Jun 10 10:13)
>> 1798299 (Yes) - ham-net-darxus.log (Jun 10 10:13)
>> 1798299 (Yes) - ham-net-thendrikx.log (Jun 10 09:25)
>> 
>> 6/6 matches (3 ham, 3 spam)
>> 
>> WARNING: NOT ENOUGH CONTRIBUTORS, MINIMUM REQUIRED IS 10!!!
>> 
>> Recent masscheck submission listings in the past week:
>> SVN rev (Match) File Name (Date)
>> 1797477  (No) - spam-net-axb-ninja.log (Jun 3 12:53)
>> 1797477  (No) - spam-net-ena.log (Jun 3 15:36)
>> 1797477  (No) - spam-net-jbrooks.log (Jun 3 13:56)
>> 1797476  (No) - spam-net-kgolding.log (Jun 3 12:40)
>> 1797477  (No) - ham-net-jarif.log (Jun 3 17:09)
>> 1797477  (No) - ham-net-axb-ham-misc.log (Jun 3 12:53)
>> 1797477  (No) - ham-net-axb-coi-bulk.log (Jun 3 12:53)
>> 1797477  (No) - spam-net-axb-generic.log (Jun 3 12:53)
>> 1797477  (No) - ham-net-axb-ninja.log (Jun 3 12:53)
>> 1797477  (No) - ham-net-jbrooks.log (Jun 3 13:56)
>> 1797476  (No) - ham-net-kgolding.log (Jun 3 12:40)
>> 1797477  (No) - spam-net-axb-ham-misc.log (Jun 3 12:53)
>> 1797477  (No) - spam-net-jarif.log (Jun 3 17:09)
>> 1797477  (No) - spam-net-axb-coi-bulk.log (Jun 3 12:53)
>> 1797477  (No) - ham-net-ena.log (Jun 3 15:36)
>> 1797477  (No) - ham-net-axb-generic.log (Jun 3 12:53)
>> 
>> 0/16 matches (0 ham, 0 spam)
>> 
>> NOTE: Old are probably no longer running automasscheck script.
>> 

Re: Tracking tasks/work for sysadmins

[Kevin A. McGrail]

On 6/8/2017 12:56 PM, Bryan Vest wrote:

Another thought if it sounds interesting and it is allowed by the ASF.
Setting up a slack channel for the admin group. More direct 
communication, and the webhooks could be used to notify of situations 
the admin's need to be or would like to be notified about.

The clients are supported on almost any platform.
The free version will store the last 10k messages.

We started using it where I work when communication between different 
departments started to fracture.


The ASF wouldn't care to my knowledge.  They use hipchat though like 
this for Infra https://www.hipchat.com/gOT4aZiMp


I don't know if we have a paid account, etc.

I personally don't like Slacks, IRC, Hipchat, etc.  I work on a number 
of things that require my full time and attention.  Things like this 
that bing me drive me up a wall ;-)  But don't let that stop you guys 
from using whatever works but realize I won't be a common presence there 
very much.


Regards,

KAM

Re: Mirrors

[Kevin A. McGrail]

On 6/8/2017 1:19 PM, Bryan Vest wrote:

As I read through the email's there was some talk about mirrors. Does SA
need more mirrors at this time?

We run on a 10Gb/s backbone, well multiple backbone providers that add up
to about 10Gb/s. Depending on the bandwidth/hardware requirements to run a
mirror I may be able to get one set up here. It would be in North West
Ohio. We have wave fiber to Chicago, Indianapolis, Cleveland, Toledo,
Columbus and Dayton.


Yes, it would be helpful, yes.  Many hands make light work.

The task is pretty simple and Dave can give you more info about 
bandwidth.  But in short, it's rsyncing a directory routinely (I think I 
have it running every 5 minutes), and then serving that dir at something 
like sa-update.XYZ.tld.


There is a weighting system when your mirror is online and I rarely 
notice it.  When I was one of the sole mirrors, I switched to a heavily 
modified httpd.conf that couldn't run CGI's etc. but it still wasn't 
bad.  I don't even notice it's running really.



Regards,

KAM

Re: Tracking tasks/work for sysadmins

[Kevin A. McGrail]

On 6/8/2017 12:29 PM, Bryan Vest wrote:

I also like this idea. There will always be things to do and that will help
a lot.


Welcome back!  The idea is great because I've been writing notes for 
eons about things that could be done and it will give me a place to 
document and discuss them all so it's captured in one place.

Re: Tracking tasks/work for sysadmins

[Kevin A. McGrail]

On 6/8/2017 12:05 PM, Dave Jones wrote:

Kevin,
Do we have access to something like Jira or BZ where we can put in our 
tasks for this team and track them?  I think it will be essential to 
plan out future goals and track issues/tasks for our group. 


Yes, that is a GREAT idea.

We have https://bz.apache.org/SpamAssassin/

We could add a component for Sysadmins.

Sign up for an account and I can give you admin access.  If we can make 
it email sysadmins@s.a.o about new issues opened that would be great.


Best,

KAM

Fwd: [jira] [Updated] (INFRA-14294) Increase RAM and VMs for SA-VM1

[Kevin A. McGrail]

FYI, looks good now.



 Forwarded Message 
Subject:[jira] [Updated] (INFRA-14294) Increase RAM and VMs for SA-VM1
Date:   Wed, 7 Jun 2017 18:09:18 + (UTC)
From:   Chris Lambertus (JIRA) <j...@apache.org>
To: kmcgr...@apache.org



 [ 
https://issues.apache.org/jira/browse/INFRA-14294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lambertus updated INFRA-14294:

Status: Waiting for user  (was: Waiting for Infra)

Looks that way. I did a full restart and it's coming back up now.



Increase RAM and VMs for SA-VM1
---

Key: INFRA-14294
URL: https://issues.apache.org/jira/browse/INFRA-14294
Project: Infrastructure
 Issue Type: Project
 Components: VM
   Reporter: Kevin A. McGrail
   Assignee: Chris Lambertus
   Priority: Critical

Hello Infra!  We have successfully migrated some of our backend from zones2 and 
spamassassin-vm to sa-vm1.
However, we are overloading things.  As we get ready to add more of the 
features, it will get worse.
We appear to have 2 cores and 4GB of RAM.  Can you please increase that to 6 
cores and 8GB of ram?
Also, is it possible to add space to the VM to avoid the OOM killer taking down 
daemons?




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Fwd: [jira] [Updated] (INFRA-14294) Increase RAM and VMs for SA-VM1

[Kevin A. McGrail]

Agreed and confirmed that I see the same.  I've kicked the ticket back.

On 6/7/2017 9:39 AM, Dave Jones wrote:
Normally a KVM guest would need to be shutdown completely "cold" to 
get the new resources to become active.  A "warm" reboot from the 
server itself would not do it.  I just rebooted it and confirmed we 
are still at the old allocations.  It looks like we will need someone 
from INFRA to take it completely down and start it back up to get the 
new resources.  This should be fine to do any time during the day in 
US hours since it should only be about a 1 minute or less outage.


Dave

On 06/07/2017 08:28 AM, Kevin A. McGrail wrote:
Dave, see re new cores and ram.  They don't use swap space on vms so 
let me know if more ram is needed.

Regards,
KAM


 Original Message 
From: "Chris Lambertus (JIRA)" <j...@apache.org>
Sent: June 6, 2017 11:10:18 PM EDT
To: kmcgr...@apache.org
Subject: [jira] [Updated] (INFRA-14294) Increase RAM and VMs for SA-VM1


  [ 
https://issues.apache.org/jira/browse/INFRA-14294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel 
]


Chris Lambertus updated INFRA-14294:

 Status: Waiting for user  (was: Waiting for Infra)

I bumped to 4 cores and 8GB ram. Let's see how this goes, and if you 
still have a lot of CPU contention we can look at bumping more. We do 
not add swap to VMs, it's too much of a performance killer for the 
disk subsystem. Would rather throw more RAM at it if needed, so let 
me know if you're still seeing stuff OOMkilled.





Increase RAM and VMs for SA-VM1
---

 Key: INFRA-14294
 URL: https://issues.apache.org/jira/browse/INFRA-14294
 Project: Infrastructure
  Issue Type: Project
  Components: VM
    Reporter: Kevin A. McGrail
Assignee: Chris Lambertus
Priority: Critical

Hello Infra!  We have successfully migrated some of our backend from 
zones2 and spamassassin-vm to sa-vm1.
However, we are overloading things.  As we get ready to add more of 
the features, it will get worse.
We appear to have 2 cores and 4GB of RAM.  Can you please increase 
that to 6 cores and 8GB of ram?
Also, is it possible to add space to the VM to avoid the OOM killer 
taking down daemons?




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Fwd: [jira] [Updated] (INFRA-14294) Increase RAM and VMs for SA-VM1

[Kevin A. McGrail]

Dave, see re new cores and ram.  They don't use swap space on vms so let me 
know if more ram is needed.
Regards,
KAM


 Original Message 
From: "Chris Lambertus (JIRA)" <j...@apache.org>
Sent: June 6, 2017 11:10:18 PM EDT
To: kmcgr...@apache.org
Subject: [jira] [Updated] (INFRA-14294) Increase RAM and VMs for SA-VM1


 [ 
https://issues.apache.org/jira/browse/INFRA-14294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lambertus updated INFRA-14294:

Status: Waiting for user  (was: Waiting for Infra)

I bumped to 4 cores and 8GB ram. Let's see how this goes, and if you still have 
a lot of CPU contention we can look at bumping more. We do not add swap to VMs, 
it's too much of a performance killer for the disk subsystem. Would rather 
throw more RAM at it if needed, so let me know if you're still seeing stuff 
OOMkilled.



> Increase RAM and VMs for SA-VM1
> ---
>
> Key: INFRA-14294
> URL: https://issues.apache.org/jira/browse/INFRA-14294
> Project: Infrastructure
>  Issue Type: Project
>  Components: VM
>Reporter: Kevin A. McGrail
>Assignee: Chris Lambertus
>Priority: Critical
>
> Hello Infra!  We have successfully migrated some of our backend from zones2 
> and spamassassin-vm to sa-vm1.
> However, we are overloading things.  As we get ready to add more of the 
> features, it will get worse.
> We appear to have 2 cores and 4GB of RAM.  Can you please increase that to 6 
> cores and 8GB of ram?
> Also, is it possible to add space to the VM to avoid the OOM killer taking 
> down daemons?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Potential problem with do-stable-update-with-scores cron job

[Kevin A. McGrail]

On 6/2/2017 9:10 AM, Dave Jones wrote:
On the ruleqa list yesterday, John Hardin confirmed that buildbot was 
running before to centrally process the full uploaded corpora.  Sigh. 
This is going to be difficult to figure out and get going again so we 
need to focus on the distributed masscheck contributions first which 
is close to being good.


The http://ruleqa.spamassassin.org site is not going to be updating 
until we figure out what was running that. 


Understood.  Yes, we had  system that ran on uploaded corpora. I think 
that is significantly less priority than our other rules process.


In effect, it was an rsync upload and masscheck system.

I personally think that sa-vm1 will not be anywhere near fast enough to 
handle the load.


Regards,

KAM

Re: Questions about build/mkupdates/run_part2

[Kevin A. McGrail]

On 5/30/2017 8:46 PM, Dave Jones wrote:



Only in old/: 72_scores.cf
Only in old/: languages
Only in old/: MIRRORED.BY
Only in old/: sa-update-pubkey.txt
Only in old/: STATISTICS-set0-72_scores.cf.txt
Only in old/: STATISTICS-set1-72_scores.cf.txt
Only in old/: STATISTICS-set2-72_scores.cf.txt
Only in old/: STATISTICS-set3-72_scores.cf.txt
Only in old/: user_prefs.template

See attached full diff output.


Interesting... I show the files existed... Perhaps I'm mixing 3.4 and 
4.0 rulesets?


Do you have any thoughts if 1786855.tar.gz (the last before the server 
went down) is 3.4 or 4.0?


Regards,
KAM


--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: Potential problem with do-stable-update-with-scores cron job

[Kevin A. McGrail]

I think you are right.  And that the system needs a run to create the 
files for the scores on the new server.


On 5/30/2017 8:25 PM, Kevin A. McGrail wrote:

It's quite possible bb does update rule qa.  Don't take what I say as 
absolute...  Will look at this more tomorrow.
Regards,
KAM

On May 30, 2017 8:15:16 PM EDT, Dave Jones <da...@apache.org> wrote:

If you look on the sa-vm1 box in /etc/cron.d/automc you will see all of

the cron jobs that I found from the /home/kmcgrail/SAcron mbox.  I
updated the bottom of the InfraNote2017 wiki page with what these cron
jobs do.  I think this is the core of the ruleqa process that allows us

to validate rules from the submitted corpus and start sa-update working

again but the http://ruleqa.spamassassin.org site is not getting
updated.  This is my next thing to tackle.  Now that I know buildbot
wasn't involved that will same me a lot of time.  I am concerned that
where ever this part was running wasn't backed up. I may have to take
your advice and post on the dev mailing list to see if anyone knows
anything or has some backups somewhere.

Dave

On 05/30/2017 07:00 PM, Kevin A. McGrail wrote:

Ahh. This was for the quick masschecks for the preflight rules. I
don't think it is used nor is the corpora it is run against

maintained.

Is there parts of rule qa that aren't working other than these quick
checks?
Regards,
KAM

On May 30, 2017 7:50:40 PM EDT, Dave Jones <da...@apache.org> wrote:

 In looking into what updates the RuleQAApp web interface for
 http://ruleqa.spamassassin.org, I found this:

 https://wiki.apache.org/spamassassin/PreflightBuildBot

 Something has to do some processing to update the data folder

that the

 ruleqa.cgi uses for the http://ruleqa.spamassassin site.  If we

can

 simply run the commands/steps from a cron'd script, that would be

better

 but I am not finding any such script so far.  May have to create

one.

 Buildbot configs are in SVN:



https://svn.apache.org/repos/asf/spamassassin/trunk/build/buildbot/

 Dave

 On 05/30/2017 05:54 PM, Kevin A. McGrail wrote:

 Well, what are we using buildbot for? I'm trying to remember
 and it might not be needed. On 5/30/2017 5:08 PM, Dave Jones
 wrote:

 I think I have figured out the primary hurdle I was
 hitting when I wrote that last email on 5/27. I found

some

 stuff in the backups the very long and hard way to get a
 little closer. Now I am at the buildbot setup and
 discovery stage. I installed buildbot on sa-vm1 but there
 is a huge change in versions so the old master config

file

 in SVN has to be converted into the new version which

will

 take me some time as I have to learn buildbot from
 scratch. Dave On 05/30/2017 12:17 PM, Kevin A. McGrail

wrote:

 On 5/27/2017 3:12 PM, Dave Jones wrote:

 This script:


https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/do-nightly-rescore-example

 refers to
 '/home/dos/sa-score-gen/nightly-rescore-via-cron'
 which is not in any backups under
 /usr/local/spamassassin/backups. A few lines down
 in the script it refers to 'generate-new-scores'
 which exists in:


https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/

 It appears that his is a different
 'generate-new-scores' than what might have been

in

 the missing /home/dos/sa-score-gen since it's
 doing some rsync'ing to pull in corpus which
 should have been on the same server? Where the
 servers separated before? Was
 rsync.spamassassin.org
 <http://rsync.spamassassin.org> on a different

box

 than where these cron jobs ran?

 Dave, What machines ran what, splitting and
 recombining servers, etc. is too much for me to
 remember, sorry. I can try and dig too but it's
 confusing to me as well. However, an important thing
 is are we running the correct scripts. Since I don't
 see /home/dos on spamassassin-vm1 in my backups, I'm
 guessing it was on a different server. Regards, KAM

Re: Backups & Crashplan

[Kevin A. McGrail]

On 6/1/2017 9:30 AM, Dave Jones wrote:
Where should I put the private key then?  If you are going to 
personally see Greg, then it may make more sense for you to generate 
it offline so the private key is not checked into SVN or emailed from 
me to you. 
Sorry, I wasn't clear.  In my head, I had been thinking about giving him 
just the passphrase out of band.


If you generate a key pair with a ridiculously strong passphrase which 
you can relay over the phone, we can then email the private, passphrase 
protected key pair to Greg.  I'll follow-up with the passphrase in 
person.  Then once you and I confirm we have the private key off the 
server and safely onto our own network, we are safe enough I believe.


Then we should only need the public key in our key rings to encrypt it 
to that sysadmins@ account.


This matched what Greg discussed a week or 3 ago.

Regards,
KAM

Re: Backups & Crashplan

[Kevin A. McGrail]

On 5/31/2017 2:52 PM, Dave Jones wrote:

On 05/30/2017 05:44 PM, Kevin A. McGrail wrote:


We should add /etc and /var/www and exclude 
/usr/local/spamassassin/backups since it's so large.

Added/excluded as suggested. Thanks very much for the feedback


Do we need to setup crashplan to run under supervisord and have monit 
email the sysadmins if it stops running again?
Need? No, CP alerts me if it doesn't run a backup for a few days.  I 
don't know why it stopped though...
Yes. They look fine.  I can create the recovery gpg key if you want me 
to then get it over to the infra team for long-term storage.  Then we 
would need to resign everything with it plus the current sysadmins' keys. 
Yes, this would be good for June 14th when I can give the GPG key 
personally to Greg.

Re: Cron <root@sa-vm1> cd / && run-parts --report /etc/cron.hourly

[Kevin A. McGrail]

Sounds awesome and feel free to spam sysadmins@

I did not get to look at the other issue but I will.
Regards,
KAM

On May 31, 2017 7:27:53 PM EDT, Dave Jones  wrote:
>This is a script I am testing out just for notifications to my email 
>address.  I will set it up to be silent now like it should have been. 
>:)  The goal of this script is to let me know when the ruleqa 
>submissions are not lining up with the current tagged SVN version of 
>rules to be masscheck'd.  I want to make sure I have the timing correct
>
>on all of the cron jobs now.
>
>I have some ideas on how to make this hourly in the future so we could 
>speed up testing and promotion of rules fairly easily by getting the 
>masscheck submitters to run the automasscheck-minimal.sh script hourly 
>after some minor adjustments to that script.  This would require some 
>adjustments to the current cron job scripts to add a 
>tagged_build/hourly_mass_check dir to the rsync area and some tweaks to
>
>the current scripts that I have been pouring through the past month or
>so.
>
>This will be months down the road but I want to learn all of the timing
>
>and dependencies over the next few weeks to make sure I have everything
>
>covered.
>
>Dave
>
>On 05/31/2017 06:17 PM, Cron Daemon wrote:
>> /etc/cron.hourly/checkMasscheckContribs:
>> 
>> SVN tagged rev in nightly_mass_check:  1796997
>> 
>> Corpus total: 120, Old: 84, Recent: 18, New: 18
>> 
>> New masscheck submission listings:
>> SVN rev (Match) File Name (Date)
>> 1796997 (Yes) - spam-darxus.log (May 31 02:10)
>> 1796997 (Yes) - ham-darxus.log (May 31 02:10)
>> 1796997 (Yes) - ham-grenier.log (May 31 02:02)
>> 1796997 (Yes) - ham-ena.log (May 31 08:59)
>> 1796997 (Yes) - spam-axb-generic.log (May 31 04:38)
>> 1796997 (Yes) - spam-axb-ham-misc.log (May 31 04:38)
>> 1796997 (Yes) - spam-grenier.log (May 31 02:02)
>> 1796997 (Yes) - ham-axb-ham-misc.log (May 31 04:38)
>> 1796997 (Yes) - ham-axb-generic.log (May 31 04:38)
>> 1796997 (Yes) - ham-axb-ninja.log (May 31 04:38)
>> 1796997 (Yes) - spam-axb-ninja.log (May 31 04:38)
>> 1796997 (Yes) - spam-jarif.log (May 31 11:58)
>> 1796997 (Yes) - ham-thendrikx.log (May 31 02:03)
>> 1796997 (Yes) - ham-jarif.log (May 31 11:58)
>> 1796997 (Yes) - spam-axb-coi-bulk.log (May 31 04:38)
>> 1796997 (Yes) - spam-thendrikx.log (May 31 02:03)
>> 1796997 (Yes) - ham-axb-coi-bulk.log (May 31 04:38)
>> 1796997 (Yes) - spam-ena.log (May 31 08:59)
>> 
>> 18/18 matches
>> run-parts: /etc/cron.hourly/checkMasscheckContribs exited with return
>code 1
>> 

Re: Potential problem with do-stable-update-with-scores cron job

[Kevin A. McGrail]

It's quite possible bb does update rule qa.  Don't take what I say as 
absolute...  Will look at this more tomorrow. 
Regards,
KAM

On May 30, 2017 8:15:16 PM EDT, Dave Jones <da...@apache.org> wrote:
>If you look on the sa-vm1 box in /etc/cron.d/automc you will see all of
>
>the cron jobs that I found from the /home/kmcgrail/SAcron mbox.  I 
>updated the bottom of the InfraNote2017 wiki page with what these cron 
>jobs do.  I think this is the core of the ruleqa process that allows us
>
>to validate rules from the submitted corpus and start sa-update working
>
>again but the http://ruleqa.spamassassin.org site is not getting 
>updated.  This is my next thing to tackle.  Now that I know buildbot 
>wasn't involved that will same me a lot of time.  I am concerned that 
>where ever this part was running wasn't backed up. I may have to take 
>your advice and post on the dev mailing list to see if anyone knows 
>anything or has some backups somewhere.
>
>Dave
>
>On 05/30/2017 07:00 PM, Kevin A. McGrail wrote:
>> Ahh. This was for the quick masschecks for the preflight rules. I 
>> don't think it is used nor is the corpora it is run against
>maintained.
>>
>> Is there parts of rule qa that aren't working other than these quick 
>> checks?
>> Regards,
>> KAM
>>
>> On May 30, 2017 7:50:40 PM EDT, Dave Jones <da...@apache.org> wrote:
>>
>> In looking into what updates the RuleQAApp web interface for
>> http://ruleqa.spamassassin.org, I found this:
>>
>> https://wiki.apache.org/spamassassin/PreflightBuildBot
>>
>> Something has to do some processing to update the data folder
>that the
>> ruleqa.cgi uses for the http://ruleqa.spamassassin site.  If we
>can
>> simply run the commands/steps from a cron'd script, that would be
>better
>> but I am not finding any such script so far.  May have to create
>one.
>>
>> Buildbot configs are in SVN:
>>
>>
>https://svn.apache.org/repos/asf/spamassassin/trunk/build/buildbot/
>>
>> Dave
>>
>> On 05/30/2017 05:54 PM, Kevin A. McGrail wrote:
>>
>> Well, what are we using buildbot for? I'm trying to remember
>> and it might not be needed. On 5/30/2017 5:08 PM, Dave Jones
>> wrote:
>>
>> I think I have figured out the primary hurdle I was
>> hitting when I wrote that last email on 5/27. I found
>some
>> stuff in the backups the very long and hard way to get a
>> little closer. Now I am at the buildbot setup and
>> discovery stage. I installed buildbot on sa-vm1 but there
>> is a huge change in versions so the old master config
>file
>> in SVN has to be converted into the new version which
>will
>> take me some time as I have to learn buildbot from
>> scratch. Dave On 05/30/2017 12:17 PM, Kevin A. McGrail
>wrote:
>>
>> On 5/27/2017 3:12 PM, Dave Jones wrote:
>>
>> This script:
>>
>https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/do-nightly-rescore-example
>> refers to
>> '/home/dos/sa-score-gen/nightly-rescore-via-cron'
>> which is not in any backups under
>> /usr/local/spamassassin/backups. A few lines down
>> in the script it refers to 'generate-new-scores'
>> which exists in:
>>
>https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/
>> It appears that his is a different
>> 'generate-new-scores' than what might have been
>in
>> the missing /home/dos/sa-score-gen since it's
>> doing some rsync'ing to pull in corpus which
>> should have been on the same server? Where the
>> servers separated before? Was
>> rsync.spamassassin.org
>> <http://rsync.spamassassin.org> on a different
>box
>> than where these cron jobs ran?
>>
>> Dave, What machines ran what, splitting and
>> recombining servers, etc. is too much for me to
>> remember, sorry. I can try and dig too but it's
>> confusing to me as well. However, an important thing
>> is are we running the correct scripts. Since I don't
>> see /home/dos on spamassassin-vm1 in my backups, I'm
>> guessing it was on a different server. Regards, KAM
>>

Re: Potential problem with do-stable-update-with-scores cron job

[Kevin A. McGrail]

Well, what are we using buildbot for? I'm trying to remember and it 
might not be needed.


On 5/30/2017 5:08 PM, Dave Jones wrote:
I think I have figured out the primary hurdle I was hitting when I 
wrote that last email on 5/27.  I found some stuff in the backups the 
very long and hard way to get a little closer.


Now I am at the buildbot setup and discovery stage.  I installed 
buildbot on sa-vm1 but there is a huge change in versions so the old 
master config file in SVN has to be converted into the new version 
which will take me some time as I have to learn buildbot from scratch.


Dave

On 05/30/2017 12:17 PM, Kevin A. McGrail wrote:

On 5/27/2017 3:12 PM, Dave Jones wrote:

This script:

https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/do-nightly-rescore-example 



refers to '/home/dos/sa-score-gen/nightly-rescore-via-cron' which is 
not in any backups under /usr/local/spamassassin/backups.


A few lines down in the script it refers to 'generate-new-scores' 
which exists in:


https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/ 



It appears that his is a different 'generate-new-scores' than what 
might have been in the missing /home/dos/sa-score-gen since it's 
doing some rsync'ing to pull in corpus which should have been on the 
same server? Where the servers separated before?  Was 
rsync.spamassassin.org on a different box than where these cron jobs 
ran?



Dave,

What machines ran what, splitting and recombining servers, etc. is 
too much for me to remember, sorry.  I can try and dig too but it's 
confusing to me as well.


However, an important thing is are we running the correct scripts. 
Since I don't see /home/dos on spamassassin-vm1 in my backups, I'm 
guessing it was on a different server.


Regards,
KAM

Re: Potential problem with do-stable-update-with-scores cron job

[Kevin A. McGrail]

On 5/27/2017 3:12 PM, Dave Jones wrote:

This script:

https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/do-nightly-rescore-example 



refers to '/home/dos/sa-score-gen/nightly-rescore-via-cron' which is 
not in any backups under /usr/local/spamassassin/backups.


A few lines down in the script it refers to 'generate-new-scores' 
which exists in:


https://svn.apache.org/repos/asf/spamassassin/trunk/masses/rule-update-score-gen/ 



It appears that his is a different 'generate-new-scores' than what 
might have been in the missing /home/dos/sa-score-gen since it's doing 
some rsync'ing to pull in corpus which should have been on the same 
server? Where the servers separated before?  Was 
rsync.spamassassin.org on a different box than where these cron jobs ran?



Dave,

What machines ran what, splitting and recombining servers, etc. is too 
much for me to remember, sorry.  I can try and dig too but it's 
confusing to me as well.


However, an important thing is are we running the correct scripts.  
Since I don't see /home/dos on spamassassin-vm1 in my backups, I'm 
guessing it was on a different server.


Regards,
KAM




--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: What a mess!

[Kevin A. McGrail]

On 5/28/2017 5:29 PM, Dave Jones wrote:
I have been all day on the do-stable-update-with-scores script.  There 
is a build problem now that has me stumped.  All of these scripts are 
a mess!  They all do similar things in very different ways making it 
very hard to follow and troubleshoot.


I found a new-rule-score-gen directory in the backup under 
/home/updatesd/svn that is definitely not checked into the SA SVN. 
Maybe this will help me get past the part that is failing. 


Agreed.  Thank you for the time and energy on fixing this. Responding to 
your other email in a moment.

Backups & Crashplan was Re: What a mess!

[Kevin A. McGrail]

On 5/28/2017 5:29 PM, Dave Jones wrote:
I hope this server is being backed up after all of this time spent on 
getting everything working again. 


Your question asking about backups was important because no, I don't 
know what backups exist of the machines.  Based on previous experience, 
there are none.  So I have run Crashplan for just this reason.


Additionally, I got a notice CP wasn't running since the 26th so I just 
started the service again. i.e. service crashplan start


I've added the credentials for the crashplan service to sysadmins/accounts.

*IMPORTANT: we are backing up /root, /home and /usr/local.  Is there 
anything else we should be backing up?

*

**
Finally, could you look this over and add it to InfraNotes2017?


Crashplan

Crashplan is sometimes used to backup SpamAssassin project machines as 
an additional safety valve.


Credentials: 
https://svn.apache.org/repos/asf/spamassassin/sysadmins/accounts/crash.pccc.com.enc.README


To configure the client, choose what gets backed up, etc., you have to 
interface with the client.  When working with remote machines, you can 
install the client locally and use SSH to port forward to remotely 
administer things.  They call this managing a headless box.


See 
https://support.code42.com/CrashPlan/4/Configuring/Use_CrashPlan_on_a_headless_computer_version_4.2_and_earlier 
for more details.



Regards,

KAM


--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: SA updates files archive

[Kevin A. McGrail]

On 5/30/2017 9:57 AM, Dave Jones wrote:
I setup an archive dir under the updates dir that is excluded from 
rsync so we would still keep the files on the server but they would 
drop off of the rsync out to the mirrors.


I haven't moved any files yet.


What I'd suggest is:

- See my "soon to be sent email" about crashplan

- Open a ticket with Infra and find out about redundancy/backups for sa-vm1.

- Only once we have sufficient backups, change the rsync as you describe 
above.


Regards,

KAM

Re: SA updates files archive

[Kevin A. McGrail]

On 5/30/2017 9:47 AM, Dave Jones wrote:

Ok.  That's fine.
To be clear, if we can confirm better backup situation, the archive dir 
idea sounds great!


Working through your other emails.  I was camping with BSA for Memorial 
Day weekend.


Regards,
KAM


On 05/30/2017 08:23 AM, Kevin A. McGrail wrote:

Hi Dave,

Those files are official project release artifacts so yes, we need to 
keep them.  And the update mirrors have served as our backups of 
those files.


I'd prefer it stay as is.

Regards,
KAM

On 5/28/2017 10:59 AM, Dave Jones wrote:

Kevin,

Do we need to keep rule update files back to 2007?  It seems like 
having only the past year or even few months is all that we need. 
Technically, the only ones that need to be in the updates directory 
are ones that are pointed to by the [reverse 
version].updates.spamassassin.org TXT DNS records.


root@sa-vm1:/var/www/bbmass.spamassassin.org/updates# ll *.tar* | wc -l
4347

It's not a bid deal to leave things as they are but it doesn't make 
sense to keep them around in the SA mirrors if they are never going 
to be used by sa-update.


We would need to keep:

752902.tar.gz* for 3.1.*
895075.tar.gz* for 3.2.*
1786853.tar.gz* for 3.3.3 and greater

I would like to move all other versions of files to an archive 
subdirectory that is excluded in the rsyncd.conf that are older than 
1 month if that is OK with everyone else.  This could become 
important if we start building new rules more than once a day.








--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: > The machines that run sa-update stuff are offline at the moment.

[Kevin A. McGrail]

Frank, I think it is a good idea.  I am asking the PMC as we follow things 
pretty formally because you will have access to private data.
Regards,
KAM

On May 20, 2017 2:44:13 PM EDT, Frank Urban <m...@frank-urban.de> wrote:
>Hi Kevin,
>
>if there will be a way how I can help I will do this.
>This will give me the chance to give something back for using open 
>source in my bank for many years without paying a cent.
>I'm not a 100% crack on anything but I have good knowledge in a lot of 
>products.
>Postfix, Spamassassin, Nagios, Mysql/MariaDB, Joomla, Networking (Cisco
>
>Routers)
>The last thing I have done with Windows was my MCP on NT4. The last 
>Windows I used was Windows7. So here I can't help too much.
>
>Greetings
>
>Frank
>
>Am 20.05.2017 um 16:26 schrieb Kevin A. McGrail:
>> On 5/18/2017 2:35 PM, Frank Urban wrote:
>>> Sounds like a job where I can help. I have noch 17 years of 
>>> experience with Linux (SuSE, RH, Ubuntu, Debian) and before I worked
>
>>> with Solaris.
>>> Today I'm working as sysadmin for a large bank and managing the 
>>> Linux/Postfix based email backbone
>>> Just migrated a old SuSE server to Debian yesterday.
>>> ..but I'm not a programmer
>> Hi Frank,
>>
>> We are concerned there will be too many cooks in the kitchen trying
>to 
>> get that issue resolved.  Are you interested in helping out on the 
>> sysadmin team longer term and on different tasks, however?
>>
>> Regards,
>> KAM
>>

Re: sa-update mirror addition

[Kevin A. McGrail]

I think there is a leap missing, sorry.

Ok, so mirrored.by is in svn in site/.  The copy for rsync should perhaps be a 
checkout with the extra files on top.  Then an svn up keeps things in sync.

Mirrored.by should be cleared.up how to update.

On 5/22/2017 8:15 PM, Dave Jones wrote:

I understand the concept of the rest of the steps below but how does this link 
what is in SVN to what is actually hosted at 
http://spamassassin.apache.org/updates/MIRRORED.BY?  Maybe I am missing 
something... 

Am I overthinking this?  I am not seeing the connection to what is on 
sa-vm1.apache.org::updates and spamassassin.apache.org/updates. 



Regards,
KAM

Re: Question about sabuildtools/bin

[Kevin A. McGrail]

On 5/21/2017 9:36 PM, Dave Jones wrote:
I can replace them if that is the direction you want to go.  They look 
like old versions of perl and GnuPG.  I will start with what the 
script is actually calling from that bin directory and try to replace 
with system versions from packages.


Agreed.  That's what I've been doing with the build process to run on 
newer systems.

Re: Questions about build/mkupdates/run_part2

[Kevin A. McGrail]

On 5/21/2017 9:45 PM, Dave Jones wrote:
It does look like run_part2 was working from cron so I guess I will 
continue to do the same thing.  Maybe someone removed the passphrase 
from the gpgkey so it could be completely automated and didn't update 
the script description in the heading.


That would be my guess.

Re: Question about sabuildtools/bin

[Kevin A. McGrail]

I have been trying to.get rid of sabintools.  It is a collection of many 
different items so you might have to look exe by exe and ask me questions
 
Years ago I got all the items in sabuildtools working on a developer box of 
mine.  It might be our only hope.
Regards,
KAM

On May 21, 2017 6:00:44 PM EDT, Dave Jones  wrote:
>Kevin,
>
>This script was in the old server cron and it adds sabuildtools/bin to 
>the PATH.
>
>http://svn.apache.org/repos/asf/spamassassin/trunk/build/mkupdates/do-stable-update-with-scores
>
>When I look at an old backup the sabuildtools/bin is a bunch of
>symlinks 
>to /local/perl586/bin.  Does this specific version of perl need to be 
>back on the server and in the PATH for a build?  Can the system perl be
>
>used or do you want this particular version of perl?
>
>If we need it back, then I was going to put it in /usr/local/perl586 if
>
>that is OK.
>
>--
>Dave Jones

Fwd: Re: [Ticket#954980087406] SpamAssassin update mirror hosting at http://sa-update.secnap.net

[Kevin A. McGrail]

Woot!



 Forwarded Message 
Subject: 	Re: [Ticket#954980087406] SpamAssassin update mirror hosting 
at http://sa-update.secnap.net

Date:   Thu, 18 May 2017 13:40:42 -0400
From:   SECNAP Network Security 
Organization:   SECNAP Network Security
To: da...@apache.org
CC: kevin.mcgr...@mcgrail.com



Hello da...@apache.org,
The Ticket:[954980087406] is now considered *RESOLVED*.

If further support is required, please feel free to reply to this 
message or raise a new issue by emailing supp...@secnap.com.

--
SECNAP Network Security
US: +1 561 999 5000
Toll Free: +1 844 NET SECU (638 7328)
https://www.secnap.com

05/18/2017 09:01 - SECNAP Network Security wrote:
Hello da...@apache.org,

Could you please attempt this again?  I think I may have found the issue.
--
SECNAP Network Security
US: +1 561 999 5000
Toll Free: +1 844 NET SECU (638 7328)
https://www.secnap.com

05/18/2017 04:43 - SECNAP Network Security wrote:
Hello Dave Jones ,

There is indeed a NAT rule in place to allow this traffic.  It is 
allowing TCP(only) traffic from anywhere to port 80.  I am unable to 
locate something that would be stopping your traffic.


As you mentioned previously, it seems that traffic from other sources is 
working as expected.  In my testing, I am yet to find any other location 
that has issue connecting.  Our company has machines deployed globally, 
and I've tested from many, many countries/continents.

--
SECNAP Network Security
US: +1 561 999 5000
Toll Free: +1 844 NET SECU (638 7328)
https://www.secnap.com

05/17/2017 14:20 - Dave Jones wrote:
More information.  I ran this past one of our networking gurus at my day
job and he noticed something.  I am able to ping 204.89.241.1.

root@sa-vm1:# ping 204.89.241.1
PING 204.89.241.1 (204.89.241.1) 56(84) bytes of data.
64 bytes from 204.89.241.1: icmp_seq=1 ttl=246 time=105 ms
64 bytes from 204.89.241.1: icmp_seq=2 ttl=246 time=105 ms
64 bytes from 204.89.241.1: icmp_seq=3 ttl=246 time=105 ms
64 bytes from 204.89.241.1: icmp_seq=4 ttl=246 time=105 ms

Since the 204.89.241.0/24 network is yours, this means the traffic is
getting into your network and this is not a routing problem.

A working connection goes through 204.89.241.175 just before the final
destination of 204.89.241.6.  My guess is there is some filtering
happening on the 204.89.241.175 device.

The purpose of this sa-update.secnap.net server should be a world-wide
mirror for SpamAssassin updates.

Thanks,
Dave


On 05/17/2017 11:30 AM, Dave Jones wrote:
> Since the sa-update.secnap.net server seems to have a private IP, is
> this a port 80 forward inbound from the Internet?  Are there any ACLs
> that would limit the source connections?
>
> This is the traceroute from the other direction showing asymetric
> routing but I don't see any problems since both directions are getting
> to/from the Peak10 Florida site where the sa-update.secnap.net server is
> hosted.
>
>   1 garl.apache.org (163.172.22.164) 0.100 ms 0.091 ms 0.084 ms
>   2 163-172-22-1.rev.poneytelecom.eu (163.172.22.1) 0.315 ms 0.410 ms
> 0.455 ms
>   3 195.154.1.226 (195.154.1.226) 0.732 ms 195.154.1.228 (195.154.1.228)
> 0.709 ms 195.154.1.226 (195.154.1.226) 0.887 ms
>   4 * lag-110.ear3.Paris1.Level3.net (212.3.235.197) 1.321 ms 1.510 ms
>   5 NTT-level3-100G.Paris1.Level3.net (4.68.73.66) 1.713 ms 1.616 ms
> 1.762 ms
>   6 NTT-level3-100G.Paris1.Level3.net (4.68.73.66) 1.754 ms
> ae-2.r25.londen12.uk.bb.gin.ntt.net (129.250.6.13) 8.731 ms 8.075 ms
>   7 ae-2.r25.londen12.uk.bb.gin.ntt.net (129.250.6.13) 8.724 ms
> ae-1.r24.londen12.uk.bb.gin.ntt.net (129.250.2.26) 8.422 ms 8.389 ms
>   8 ae-5.r24.nycmny01.us.bb.gin.ntt.net (129.250.2.18) 86.796 ms 86.861
> ms ae-1.r24.londen12.uk.bb.gin.ntt.net (129.250.2.26) 8.486 ms
>   9 ae-5.r24.nycmny01.us.bb.gin.ntt.net (129.250.2.18) 86.834 ms
> ae-1.r25.nycmny01.us.bb.gin.ntt.net (129.250.3.207) 86.074 ms 86.353 ms
> 10 ae-9.r22.asbnva02.us.bb.gin.ntt.net (129.250.2.149) 86.263 ms
> ae-1.r25.nycmny01.us.bb.gin.ntt.net (129.250.3.207) 83.881 ms 79.049 ms
> 11 ae-9.r22.asbnva02.us.bb.gin.ntt.net (129.250.2.149) 92.510 ms 91.865
> ms 86.011 ms
> 12 ae-1.r20.miamfl02.us.bb.gin.ntt.net (129.250.2.87) 117.062 ms 112.294
> ms ae-0.r23.asbnva02.us.bb.gin.ntt.net (129.250.3.85) 86.724 ms
> 13 ae-1.r20.miamfl02.us.bb.gin.ntt.net (129.250.2.87) 113.779 ms 118.493
> ms ae-1.r05.miamfl02.us.bb.gin.ntt.net (129.250.2.185) 112.849 ms
> 14 ae-2.a01.miamfl02.us.bb.gin.ntt.net (129.250.3.167) 113.289 ms
> 113.269 ms ae-1.r05.miamfl02.us.bb.gin.ntt.net (129.250.2.185) 117.155 ms
> 15 ae-2.a01.miamfl02.us.bb.gin.ntt.net (129.250.3.167) 116.922 ms
> xe-0-0-24-0.a01.miamfl02.us.ce.gin.ntt.net (157.238.179.66) 117.944 ms
> ae-2.a01.miamfl02.us.bb.gin.ntt.net (129.250.3.167) 113.677 ms
> 16 xe-0-0-24-0.a01.miamfl02.us.ce.gin.ntt.net (157.238.179.66) 113.994
> ms te2-4.dist02.fll.peak10.net (96.46.240.62) 104.961 ms
> xe-0-0-24-0.a01.miamfl02.us.ce.gin.ntt.net 

Fwd: [auto] do-nightly-rescore-example 127

[Kevin A. McGrail]

Dave, there should have been cron logs as well but I didn't see them 
come into moderation.


Regards,

KAM



 Forwarded Message 
Subject:[auto] do-nightly-rescore-example 127
Date:   Tue, 16 May 2017 02:25:02 -0700 (PDT)
From:   UpdatesD Cron 
Reply-To:   rul...@spamassassin.apache.org



Exit Status 127 is not zero for do-nightly-rescore-example.

Re: sa-update mirror addition

[Kevin A. McGrail]

You should be able to edit mirrored by and good to go.  Might want to start out 
with a low weight.

We should also try and svn that info without the actual updates.
Regards,
KAM

On May 15, 2017 6:58:30 PM EDT, Dave Jones  wrote:
>Have setup sa-update.ena.com and they are ready to be checked and added
>
>to the MIRRORED.BY file.
>-- 
>Dave

Re: PowerDNS web interface

[Kevin A. McGrail]

Is lp available for projects?  

And/Or do you envision we create a key for say sysadmins@s.a.o and give you the 
private key and also a passphrase out if band.  Then we add sysadmins@s.a.o to 
any thing we encrypt as a recipient and that is a safety valve?
Regards,
KAM

On May 15, 2017 6:35:56 PM EDT, Greg Stein <gst...@gmail.com> wrote:
>We currently keep many credentials in LastPass (*). ... If y'all would
>like
>to construct a recovery key for SA, then we'll happily store that into
>the
>ASF LastPass account.
>
>Cheers,
>-g
>
>(*) after a couple LP security notices, we are considering other
>options,
>but that's neither here/there. if we switch vault providers in six
>months... we'll *still* have one for an SA recovery key.
>
>
>On Mon, May 15, 2017 at 5:27 PM, Kevin A. McGrail
><kevin.mcgr...@mcgrail.com
>> wrote:
>
>> Greg,
>>
>> Dave Jones brings up a good point about longevity of encrypted things
>for
>> the foundation.  Could infra maintain a key that can be added to
>things for
>> a backdoor?
>>
>> See below for a snapshot of the relevant thread for background.
>>
>> Regards,
>> KAM
>>
>> KAM:
>> What you should do is use the pub key at http://people.apache.org/~
>> kmcgrail/ and encrypt a file with the password.  Ideally,
>you
>> already have a key for me that chains to a circle of trust so you
>know for
>> sure it's me.  They actually have key signing parties and stuff for
>this.
>> I've found it to be a PITA and doesn't make me feel better that the
>key is
>> valid.  It's not like we are trained in verifying fake IDs so it's
>nothing
>> but an illusion of trust.
>>
>> Dave: My concern is I can sign it with your (Kevin's) key and even
>Brian's
>> key so the two of you can open it but what happens if another 5 or 10
>years
>> go by and we 3 are no longer volunteering as SA sysadmins?  The next
>> generation of sysadmins won't be able to open these files.
>>
>> There has to be a better way where we use an encrypted file with a
>master
>> password that we share and is recorded in a save place for the
>future.
>>
>> I use LastPass for this and I have my master password in an envelope
>in a
>> safe for my wife to open in the event I am no longer on this planet.
>I have
>> instructed her to take this envelope to any of my techie friends and
>they
>> would know how to help her get access of all of my online accounts. 
>We
>> need something like this for this team.
>>
>> KAM: The first consideration is that the method above with SVN is
>> considered acceptable to the foundation and exists already.  It long
>> predates me and has a strong encryption pedigree.  It also doesn't
>rely on
>> a service being in business since it uses all open source software
>and
>> files that you can mirror today.
>>
>> What I have done that is similar to what you describe is that my
>> passphrase for my private key is in my safe.  So should I leave this
>mortal
>> coil, the data is all recoverable.
>>
>> Also, we are trying to move away from master passwords as much as
>> possible.  Sharing of root credentials should be avoided as just a
>general
>> security mantra.
>>
>> KAM: Do you feel strongly enough about it to debate it with infra and
>see
>> what their thoughts are?
>>
>> Dave: Not that strongly.  I will be glad to go along with the
>existing
>> standards.  Seems like there should be an escrow-ed key from the
>foundation
>> or something that we would also sign with for the future.
>>
>>

Re: PowerDNS web interface

[Kevin A. McGrail]

Hi Bryan,

A) My default answer is always going to be add it to the wiki with 
sensitive portions redacted and point to SVN files that are encrypted.  
This follows in kind to how extremely, sensitive items


B) In my line of work, it is absolutely a failure of any security audit 
to use a default password.  It also shouldn't be written even on this list.


What you should do is use the pub key at 
http://people.apache.org/~kmcgrail/ and encrypt a file with the 
password.  Ideally, you already have a key for me that chains 
to a circle of trust so you know for sure it's me. They actually have 
key signing parties and stuff for this.  I've found it to be a PITA and 
doesn't make me feel better that the key is valid.  It's not like we are 
trained in verifying fake IDs so it's nothing but an illusion of 
trust.


Dave, can you decrypt 
https://svn.apache.org/repos/asf/spamassassin/sysadmins/accounts/example.enc? 
There is a example.enc.README to help explain more of the process.


*Reminder: *Bryan, you need to get your public key on 
http://people.apache.org/~bvest/


Regards,
KAM

On 5/15/2017 4:01 PM, Dave Jones wrote:
I setup nesedit and wanted to pass this along.  We can put this in the 
wiki after we have properly vetted any security issues but I think I 
have it pretty secure.


1. Open an SSH tunnel from your desktop:

ssh -f sa-vm1.apache.org -L 8090:localhost:8090 -N

2. Open http://localhost:8090 from your desktop browser

3. Login with admin/admin

Do we need to change the default admin password?  My thought was it's 
not externally accessible (port 8090 listens on 127.0.0.1 and this 
port is not opened on the local firewall) and everyone on the server 
is trusted by SSH keys and has root access anyway.


I think it's secure from the outside and the default admin password is 
fine in this case.

Re: ApacheCon 2017

[Kevin A. McGrail]

Thanks.  I think getting 3.4.2 will be my goal.  I still have a lot of little 
issues to fix.
Regards,
KAM

On May 15, 2017 8:40:11 AM EDT, Dave Jones  wrote:
>Kevin,
>Good luck this week in Miami.  I think you were trying to get SA 4.0 
>built, released, and announced so I hope all goes well.
>-- 
>Dave

Re: ruleqa.spamassassin.org

[Kevin A. McGrail]

What server and path? I will see if my backups are better
Regards,
KAM

On May 14, 2017 3:00:49 PM EDT, Dave Jones  wrote:
>Nevermind.  I figured it out.  I needed to have some data files for the
>
>script to read.  It's working now showing the last backup we had on Jan
>
>26th.
>
>Now I am working on getting the backend scripts to run...
>
>Dave
>
>On 05/14/2017 10:42 AM, Dave Jones wrote:
>> Kevin,
>> http://ruleqa.spamassassin.org/
>> 
>> Software error:
>> 
>> Can't use an undefined value as an ARRAY reference at 
>> /usr/local/spamassassin/svn/trunk/masses/rule-qa/automc/ruleqa.cgi
>line 
>> 725.
>> 
>> I am stuck on a perl issue.  I tried to get the config file in the
>same 
>> dir setup properly to make sure that wasn't the problem.  I am not
>good 
>> enough with perl to troubleshoot this one much further.
>> 

Re: SysAdmin Tasklist was Re: Next priority to get running on sa-vm1.apache.org

[Kevin A. McGrail]

On 5/14/2017 10:11 AM, Dave Jones wrote:


Do we want to subscribe root like this?  It doesn't need to receive 
any of these emails that will just fill up the root mailbox or 
possibly create a mail loop.  I was thinking about allowing it as a 
non-member poster.  I am more familiar with Mailman that allows a list 
of addresses that can post without being a member.  Do you know if 
there is something like this available on the ASF lists?  If so, I 
assumed this would be a Jira task for someone with admin rights to the 
listserv.




Good point.  Can you open an infra ticket and ask?  In the meantime just 
subscribe root.


I tweaked the sa-update-mirror-check.sh script and it's now 
/usr/local/bin/checkSAupdateMirrors.sh and symlink'd to 
/etc/cron.daily to send this list an email with the status of all the 
mirrors. 


Good!  Can you change it to hourly when you get it tweaked.

Buildbot is likely a priority

[Kevin A. McGrail]

So in working on the build again, figuring out the data for this website 
- http://buildbot.spamassassin.org/updatestage//1786853.tar.gz and 
answering the what is buildbot, what is ruleqa, what is updates, etc. is 
likely next.


For the time being, I've added an updatestage dir to bbmass with the 
files I need and then overridden DNS on my local machine so the scripts 
just "think" it worked.


Regards,

KAM


--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: SysAdmin Tasklist was Re: Next priority to get running on sa-vm1.apache.org

[Kevin A. McGrail]

RESENDING: Scripts were blocked for security reasons

On 5/13/2017 4:56 PM, Dave Jones wrote:

What's the next priority now that the rsync and httpd configs are active?

I will work on the build next using this:

https://svn.apache.org/repos/asf/spamassassin/trunk/build/README


PREFACE: I'm working on the build.  If you would like to help, we need 
to coordinate first.


Here is the promised list of items I've identified.

To me, the priority would be getting ruleqa/masscheck better documented 
and back up and running would be ideal.


If we can get that system running smoother with a shorter lag to 
publishing rules, I'd like to help more with it.



DONE - Touch a file called MIRROR.CHECK in 
/var/www/bbmass.spamassassin.org/updates on SA-VM1 and test if it is 
synced to the Mirrors.  NOTE: I sync every 10 mins


- Document on the wiki that MIRRORED.BY contains the sa update mirror 
contact names.


- Get the various files for running the sa-update aka bbmass website 
into SVN.  This would NOT be the update files but likely everything else 
including the httpd.conf, MIRRORED.BY, etc.


- Get the email to root from sa-vm1 to go to sysadmins@ without 
moderation so we have cron logs, etc. archived.


- KAM to Get the passwords for crashplan for SA into sysadmins repo 
encrypted so we have multiple people who have access.


- Get the sa-update-mirror-check script (attached) running on SA-VM1 and 
emailing sysadmins@ without moderation


- Get Darxus' rule update check script (attached) running on SA-VM1 and 
emailing sysadmins@ without moderation.  See SA Dev list example: Rule 
updates are too old - 2017-05-08


- Get Darxus' check script updated for 3.4.2 and 3.3.2.

- Perhaps update the sa-update-mirror-check to use the MIRROR.CHECK with 
a timestamp to confirm it's within a reasonable period of time.


- Find out who wrote the sa-update-mirror-check (likely on the list 
archives), check the licensing on the post and hopefully ask who wrote 
it to public domain or Apache license.  Then add 
attribution/license/copyright and add it to the sysadmins repo.


- Ask Darxus' if we can repo his script as well with 
attribution/license/copyright as above


- Ask Darxus' to turn off his script that runs on his infrastructure

- Identify what we used to provide on the old servers.  Some things KAM 
believes we had that need to be verified and likely expanded on:


  o Masscheck RSYNC for people to send us their Masscheck Logs
  o An email system for people to email and it would send the results 
of checking that email
  o Masscheck Corpora RSYNC or perhaps SSH for people to send us their 
corpora for us to run our own Masscheck server.  NOTE: This is the most 
sensitive data we would have I believe since it is other people's real 
mail.
  o For the above, I think I myself have this setup.  I'd like to 
identify where and extend it / improve it / make sure it's working, etc.

  o Look at the rsync MOTD[1]
  o Masscheck stuff: 
https://wiki.apache.org/spamassassin/NightlyMassCheck - KAM sent notes a 
few days ago about how he got this running on spamassassin-vm.  if that 
doesn't suffice, please let me know.


- Identify what jm was using talon1.pccc.com to provide so I can mimic 
it.  His cron jobs were disabled last January but I think they were 
running items related to masscheck.


- Get the RuleQA Website running again.

- Identify what the incoming.spamassassin.org server did/does/can do for 
us.  NOTE: It might be the the same as below.


- Talk to Grant Kellar with Sonic about the traps they have in place and 
where they are sent to make sure we are utilizing them.


- Clean up and remove unnecessary backup data on sa-vm1 - NO NEED TO BE 
HASTY ON THIS, I'M JUST WRITING A COMPLETE LIST.


- Identify how much data we need if Infra can shrink the data storage 
allocated for sa-vm1


- Talk to AXB about SOUGHT and SOUGHT2

- Update the documentation for InfraNotes2017 with another pass of 
updates about machines, etc.



[1]
corpus
nightly mass-check result upload area.  It is password protected.
If you would like a password, please send a request to
p...@spamassassin.apache.org and request a "nightly" username and password.

submit
Score generation mass-check result upload area.  It is password
protected.  If you would like a password, please send a request to
p...@spamassassin.apache.org and request a "score generation" username
and password.  Generally these are only granted after a mass-check
announcement has been made on the spamassassin developer mailing list.

anoncorpus
mass-check result download area, available via anonymous access.


--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

#!/usr/bin/perl

# host -t txt 2.3.3.updates.spamassassin.org

use strict;
use warnings;
use Net::DNS;
use POSIX qw(strftime);
use LWP::Simple;

### Checking updates

my $updatelog = '/home/darxus/progs/sa/updateve

Re: Rsync config on sa-vm1.apache.org

[Kevin A. McGrail]

On 5/13/2017 4:47 PM, Dave Jones wrote:

It's working now.  Seeing a lot of clients syncing now in rsyncd.log.

BTW, I have setup a redundant mirror of sa-update.ena.com to add to 
the MIRRORED.BY.  I have to get some firewall ACL help on Monday to 
allow port 80 to the servers.  After that, you can check them out and 
add them to the list. 


Excellent.  It will be good to have more mirrors!

I created a MIRROR.CHECK file on the master rsync and confirmed it's 
being updated.


Saw your other email and I am writing a lot of detail onto that.

NOTE: I deleted the tar file I uploaded that was in /var/www to save space.

Regards,

KAM

Fwd: SA-Update Mirror Check: http://sa-update.secnap.net/ is up again

[Kevin A. McGrail]

NOTE, we'll likely be changing this to go to the sysadmins@s.a.o list 
since we now have that list.




 Forwarded Message 
Subject:SA-Update Mirror Check: http://sa-update.secnap.net/ is up again
Date:   Sat, 13 May 2017 17:01:12 -0400
From:   root 
To: priv...@spamassassin.apache.org

Re: Rsync config on sa-vm1.apache.org

[Kevin A. McGrail]

On 5/13/2017 3:16 PM, Dave Jones wrote:

Are you going to pop on the box and scp your tar over?  Another
option is to use https://filedrop.ena.com and send it to me 
(djo...@ena.com) that way.


The /var/www/bbmass.spamassassin.org dir is going to have most of the 
files.  It's a lot of files in there so it would be nice/speedy if you 
could rsync the delta from your mirror.


So if you move the file sa-update-html.tar.gz from kmcgrail homedir on 
sa-vm1, that's a tar of all the files.  You can use it as a definitive 
source of truth and remove the copy you have.


Here's the script I run as a mirror.  NOTE: the box I'm running the 
mirror on is a new IP but I don't think we had any restrictions.


#!/bin/sh
/usr/bin/rsync --timeout=280 -T /tmp -ta --delete 
--address=69.171.29.39  rsync.spamassassin.org::updates 
/htdocs/sa-update.pccc.com/html > /dev/null 2>&1


Regards,

KAM

Re: Encryption and Backups was Re: Onboarding, Documentation, etc.

[Kevin A. McGrail]

On 5/13/2017 10:25 AM, Dave Jones wrote:
How exactly do you want them to be stored?  I am not familiar with 
doing this.
The process I have seen used in the ASF is to use gpg to encrypt the 
files hence why one of the requests for you and Bryan was for your 
public keys to be put up on people.


I was under the impression when you told me "there were things all 
over the place that updated DNS" this could be from other servers too.
I may have over stated the issue, sorry.  As we bring things back 
online, we'll find out :-)


My concern is that in the past, we used so much of a VM machine's 
resources that it brought the machine to it's knees.  So sa-vm1 being 
ramped up could bring that same issue to light.


So if you see the legacy list of machines, there was a lot more systems 
involved.  If we can get down to just one box, it'll be simple.  But if 
we need more boxes, sobeit!


This would be good to use something like a LastPass shared note. I use 
LastPass extensively for personal and work (LastPass Enterprise). 
Agreed.  LastPass, OnePass, etc.  As Bryan comes onboard, I'll look at 
what needs to be encrypted and if it gets too much, we can look at that.


Regards,
KAM

--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: Onboarding, Documentation, etc.

[Kevin A. McGrail]

On 5/13/2017 9:50 AM, Dave Jones wrote:
I am not sure about my goal since it may be in slight conflict with 
your goal.  :)  I would like SA to be a little more toward a complete 
spam filter out of the box so people don't have to spend years 
learning all of the ins and outs to make it effective.  I understand 
that SA can't be complete out of the box for every mail environment is 
a little different but I feel that it could be improved a little more 
with some default rules.
I support you adding it.  So it's helpful for you to know where I'm 
coming from AND that I support differing goals.  We are volunteers so 
there is no necessity for a hugely unified vision.


And note that I have supported and kept masscheck/ruleqa going as best I 
could for two decades with significant resources starting even prior  to 
project under SA.  Even though I do not use the results of either system 
in production.


Will do.  I thought about leaving it out since we aren't using that 
DNS server anymore.
Brian's longtime support of this project and his visibility in the OSS 
community are more valuable than you might know.


I hope we can figure out something to get him back in the mix and keep 
him interested in helping!


This is a link in the "Hidden Slave" on the DNS Hosting table.  I you 
want it more prominent, then I can change the link to show the URL.

I'm sure it is fine and I just missed it.

Is the "incoming" server still around?  I wasn't able to resolve it 
in DNS yesterday or today.
Yes, it's named incoming.spamassassin.org.  I might have used a.o by 
accident.  BTW, can you add a.o and s.a.o to the acronyms list?


This is documented under the SVN section.  The RO and RW should be 
self explanatory to a sysadmin.

Fair enough.  Some hurdles are good to have :-)
It's not under the Workflow section but parallel with the Onboarding 
so new people would see it.  After you read the acronymns once, you 
should have them down.
It's a source of reference for all admins that lives and breaths. 
Leaving it under Onboarding implies they never need to read it again 
which might prove false.


Do you want to document stuff twice and have to maintain it in two 
places?

The steps, yes.  perhaps not the exact details

So perhaps - Email sysadmins-subscr...@spamassassin.apache.org 
changes to - Subscribe to sysadmins@spamassassin.apache.org and the 
instructions remain in the section below.


However, as evidenced by Bryan still working to onboard, simplification 
is needed.  I want to specifically hold the hand of new sysadmins at 
least to the point where they can leave the nest to fly or flop.


Regards,
KAM

Encryption and Backups was Re: Onboarding, Documentation, etc.

[Kevin A. McGrail]

On 5/12/2017 7:32 PM, Dave Jones wrote:

One thing we need to specify in more detail is the way we are going
to encrypt things in the sysadmins repo.  We don't want to put the
encryption details on the wiki per se since it's public.

The only thing I envision in the repo encrypted is passwords.

For example, the PowerDNS API key is in the pdns.local.conf file. 

I believe documenting the location of the API key in the Wiki is sufficient.

The local firewall allows port 8081 inbound from any source and the 
conf file is restricting which IPs the daemon will respond to.  I 
would like
to restrict the PowerDNS web server/API to specific source IPs 
matching the conf file for dual layers of protection. 

Good idea!
We still shouldn't document publicly the PowerDNS API key but where 
should we document that?  It will be in many scripts on servers that 
need to update DNS records so that will be a form of documentation if 
we reference the scripts on the wiki.
I don't think there are many servers that update the DNS records. If 
there are, we can talk more but I believe it's just a local script on 
that one box when we get it working.
In my opinion, referencing scripts and config files on the wiki is 
good enough for documenting sensitive information.


Agreed but there are some items like root level passwords to old boxes, 
a shared signing key, etc. that can be at least temporarily stored in 
svn encrypted.


For example, there is a box called incoming.  I have the root password.  
But I'd prefer to not use it and switch to sudo and add accounts for you 
two.


Regards,

KAM

Re: Onboarding, Documentation, etc.

[Kevin A. McGrail]

On 5/13/2017 9:13 AM, Kevin A. McGrail wrote:

On 5/12/2017 7:32 PM, Dave Jones wrote:
I have all of this information on 
https://wiki.apache.org/spamassassin/InfraNotes2017 now.  Please 
review and comment/update as needed. 


Overall, the organization and edits are very good.  Thanks for fixing 
Tenets, I knew that word looked wrong!


I also think this information should be more clearly documented as I 
forgot it :-)


Write access to the wiki is to anyone who has created a login name on 
the wiki

whose name has been added to the page
https://wiki.apache.org/spamassassin/ContributorsGroup

Write access to that page is to anyone whose wiki login name has been 
added to

https://wiki.apache.org/spamassassin/AdminGroup

--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: Onboarding, Documentation, etc.

[Kevin A. McGrail]

On 5/12/2017 7:32 PM, Dave Jones wrote:
I have all of this information on 
https://wiki.apache.org/spamassassin/InfraNotes2017 now.  Please 
review and comment/update as needed. 


Overall, the organization and edits are very good.  Thanks for fixing 
Tenets, I knew that word looked wrong!


I added a goal section with my goal.  I suggest you add your goal as 
well and I've put this on the onboarding task list.


Rather than edit directly, I wanted to ask that you do it to look at 
what I wrote and to explain why it's needed.  NOTE: Some of these are 
edited from the previous send as I look them over again.



*- Credentials:*
   - There are legacy shared credentials for elevated access on older 
machines.  These must be encrypted and stored in SVN.  The project is 
slowly moving away from these concepts.



*- Under DNS, *List Hyperreal and that it is currently offline because 
we can't get DJBDNS to transfer a record.

   Contact for HyperReal is Brian Behlendorf

- I would also add this information for Sonic in case their IPs change, etc.

https://wiki.sonic.net/wiki/Secondary_DNS_Service is the current
configuration information you will need.


I don't see this section and it will be important in the detective work 
as you read scripts.  Even I have to refer to it quite often when I come 
across legacy documentation to figure out where something points to now.


*- Project Machines*

This is a short description of the machines involved including those 
that USED to exist and why.


OLD:
- Hyperion.Apache.org - 
ftp://ftp.ist.utl.pt/apache/dev/machines.html#hyperion shows this was 
likely a solaris box that I had access to when zones died and I had to 
recover data.

- SpamAssassin.zones.apache.org - DIED - was replaced with spamassassin-vm
- SpamAssassin.zones2.apache.org - deprecated by Infra
- spamassassin-vm.apache.org - deprecated by Infra

CURRENT:
- incoming.apache.org - Donated by Sonic
- sa-vm1.apache.org - Ubuntu box to replace spamassassin-vm and zones2

- Other Aliases: buildbot, ruleqa (there might be more).

Also, this is an ASF box for all committers:

- Minotaur.apache.org aka People - This used to handle various build and 
devel related tasks.   Minotaur.apache.org for ssh (It appears that 
minotaur is not the proper server anymore.  I used home.apache.org per 
some links that Sidney sent.  (Home.apache.org and people.apache.org 
resolve to the same IP.)



*Under the standards,* I would add this as someone will ask if we can 
use XYZ
- Ubuntu?  Ubuntu is the ASF Infrastructures OS of choice. Supporting 
others is not an option at this time.



*I think this section is important *especially because it needs others 
added to it.


- How to get access to each machine:

sa-vm1.apache.org (current as of 4/28/17)
   - Open a Jira ticket with the availid of the person(s) you want to 
have access. Note if they need sudo access or not.

   - User self maintains their ssh-key at id.apache.org
   - NOTE: if sudo access was requested, run and sets up 'ortpasswd'


*I would add this:*

- Why all the boxes?

The resources for Masscheck can be very intensive on CPU, Ram and disk 
I/O intensive.  Over the years, many boxes have been consolidated, 
donated, lost, replaced, moved under ASF Infrastructure or just fell 
over and sank into the swamp.


- Some boxes are just names for other boxes
  trap-proc.spamassassin.org. Sonic has scripts set up to archive 
collected spam to that server.



*I would add this:*

SVN:
- You need access to https://svn.apache.org/repos/asf/spamassassin/ for 
sysadmin, dns and site.
- In the ASF, we use http for read-only access to a repo and https for 
read-write.  So if you are trying to checkout and modify a repo, make 
sure you are using https://


Encrypted SVN:
- If you can, document things in the Wiki at 
https://wiki.apache.org/spamassassin/DevelopmentStuff.  If something is 
sensitive, encrypt it and store it on the 
https://svn.apache.org/repos/asf/spamassassin/sysadmins repo and 
reference it on the Wiki.



*The onboarding workflow *shouldn't include the important resources or 
acronyms IMO.  Those are good for everyone which ties into...


*The onboarding workflow *should have the extra steps added back not 
just the pointer to important resources. I am making it self-fulfilled 
and self-driven and trying to have a specific set of steps done so they 
can get to work as quickly as possible.


- Once they have an Apache ID, they should:

   - SASA Member signs up for an Infra Jira account at 
https://issues.apache.org/jira/secure/Signup!default.jspa?

   - SASA Member adds an SSH public key to id.apache.org
   - Add your PGP public key.  http://people.apache.org/~kmcgrail/
   - Create an account on our Wiki
   - Email sysadmins-subscr...@spamassassin.apache.org
   - Email sysadmins@spamassassin.apache.org and ask for karma to 
access sa-vm1 with sudo access
   - Email sysadmins@spamassassin.apache.org and ask for your account 
to be added to 

Re: SVN update area

[Kevin A. McGrail]

Hi Dave,

I don't believe I ever had my own SVN credentials on the server nor do 
we have a generic SVN.  I could be wrong but I'm not sure we let scripts 
update SVN.


I think you might find that things like scripts are in SVN but not the 
data.  So much of that data you are seeing as large isn't repo'd.


It might be better if we looked at specific dirs/issues 1 by 1 and 
decide case by case.


Regards,
KAM
On 5/12/2017 7:58 PM, Dave Jones wrote:
I need to know if I should put my own SVN creds on the server or if we 
have a generic SVN ID for automated updates from scripts in cron.


I was trying to have a central SVN area under 
/usr/local/spamassassin/svn since it's so large.  Do we need to 
have separate repos to keep things in smaller directories?  I guess it 
depends on if we have general users like automc or bbmass that can 
commit to SVN in cron'd scripts.


Dave

On 05/10/2017 08:32 PM, Kevin A. McGrail wrote:

On 5/10/2017 8:49 PM, Dave Jones wrote:

Kevin,
I was wanting to setup /usr/local/spamassassin/svn as a check out 
area and a general update area.  Do you normally set it up with your 
own creds or is there a generic user that was used on the previous 
server for cron jobs to do commits?


Looking at https://svn.apache.org/viewvc/spamassassin/dns/, it 
doesn't appear that the automated entries are stored in SVN.


Regards,

KAM

Fwd: [jira] [Commented] (INFRA-14045) Setup davej with access to sa-vm1.apache.org with sudo access

[Kevin A. McGrail]

FYI that I have asked for Bryan's access to the same box at 
https://issues.apache.org/jira/browse/INFRA-14146


Regards,

KAM



 Forwarded Message 
Subject: 	[jira] [Commented] (INFRA-14045) Setup davej with access to 
sa-vm1.apache.org with sudo access

Date:   Sat, 13 May 2017 03:58:04 + (UTC)
From:   Chris Lambertus (JIRA) <j...@apache.org>
To: kmcgr...@apache.org



[ 
https://issues.apache.org/jira/browse/INFRA-14045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16009113#comment-16009113
 ]

Chris Lambertus commented on INFRA-14045:
-

ping. anything else needed here?



Setup davej with access to sa-vm1.apache.org with sudo access
-

Key: INFRA-14045
URL: https://issues.apache.org/jira/browse/INFRA-14045
Project: Infrastructure
 Issue Type: Task
 Components: ColoTasks
   Reporter: Kevin A. McGrail
   Assignee: Chris Lambertus
   Priority: Minor

Please setup davej so he can ssh to sa-vm1.apache.org after he adds his ssh pub 
key to id.apache.org
He should have sudo access as well




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Next priority to get running on sa-vm1.apache.org

[Kevin A. McGrail]

On 5/12/2017 12:34 PM, Bryan Vest wrote:
I see the step I missed.  I'll get it fixed up. 


Excellent.  When you have done that, then Dave or I need to open a Jira 
ticket.



--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: Fwd: [jira] [Updated] (INFRA-14135) Please change DNS Servers for spamassassin.org

[Kevin A. McGrail]

On 5/11/2017 12:54 PM, Dave Jones wrote:
Excellent.  Now we are all green here (ignore the MX record since we 
intentionally don't have one): https://intodns.com/spamassassin.org


In a week or two we should be able to add in ns2.ena.com after getting 
the nod from our CTO who is out of pocket for a couple of weeks. 

Perfect.

So for DNS info on the wiki, the contact information needs add 
ns2.pccc.com contact is me.  And you might add the ena info now too.


Will give feedback in a bit about next step!

Regards,
KAM

Fwd: [jira] [Updated] (INFRA-14135) Please change DNS Servers for spamassassin.org

[Kevin A. McGrail]

FYI



 Forwarded Message 
Subject: 	[jira] [Updated] (INFRA-14135) Please change DNS Servers for 
spamassassin.org

Date:   Thu, 11 May 2017 16:25:04 + (UTC)
From:   Chris Lambertus (JIRA) <j...@apache.org>
To: kmcgr...@apache.org



 [ 
https://issues.apache.org/jira/browse/INFRA-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Lambertus updated INFRA-14135:

Status: Waiting for user  (was: Waiting for Infra)

DNS server replaced as requested.


Please change DNS Servers for spamassassin.org
--

Key: INFRA-14135
URL: https://issues.apache.org/jira/browse/INFRA-14135
Project: Infrastructure
 Issue Type: Task
 Components: DNS
   Reporter: Kevin A. McGrail
   Assignee: Chris Lambertus
   Priority: Minor

Please change the DNS servers for spamassassin.org to remove ns.hyperreal.org 
and replace it with ns2.pccc.com.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: DNS hosting of spamassassin.org

[Kevin A. McGrail]

On 5/10/2017 8:41 PM, Dave Jones wrote:

Kevin,
Please go ahead and open a jira ticket to get the registrar updated to 
remove ns.hyperreal.org and replace it with ns2.pccc.com.  If I am 
allowed to do this I will but I wasn't sure if this needed to come 
from you.
ns.hyperreal.org is getting very far behind and will be handing out 
incorrect records when we get everything rolling again soon.


Roger that.   We had talked about it so it was cool for you to open a 
JIRA but I did it anyway for you: 
https://issues.apache.org/jira/browse/INFRA-14135



Regards,
KAM


--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Notes from when SpamAssassin zones box died

[Kevin A. McGrail]

 solved when corpora uploads work & reply to 
mailing list letting him know that part is restored for john hardin
ON ANOTHER TASK - 49276 - Make sure everyone active with RuleQA is on 
mailing list including corpora upload
DONE - ADDED ANYWAY - Update 
http://wiki.apache.org/spamassassin/UploadedCorpora if chmod rsync.rsync 
is required.

DONE - Setup jhardin’s account
MOVED TO ANOTHER NOTE- Install Crashplan


* Errors 2014-06-15
   Error set #2
/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores: line 364: 
at: command not found

+ cd
+ at -q n now + 16min
/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores: line 362: 
at: command not found
+ echo 
/export/home/updatesd/svn/spamassassin/build/mkupdates/tick_zone_serial

Exit Status 127 is not zero for mkupdate-with-scores
   Error Set #1
+ STATUS=0
+ set -e
+ cd ..
+ rm -rf release_3.3.3 /tmp/sa-mkupdate-6153/release_3.3.3
+ return 0
+ update_dns_record 3.3.3 1588859
+ SA_VERSION=3.3.3
+ UPDATE_REVISION=1588859
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
++ perl -pe 's/^(\d+)\.(\d+)\.(\d+)$/$3.$2.$1/'
++ echo 3.3.3
+ RVERS=3.3.3
+ DNS_RECORD='3.3.3  TXT  "1588859"'
+ echo 'DNS Record: 3.3.3  TXT  "1588859"'
DNS Record: 3.3.3  TXT  "1588859"
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.3
+ mkdir /tmp/sa-mkupdate-6153/dns-backup
mkdir: cannot create directory `/tmp/sa-mkupdate-6153/dns-backup': File 
exists

+ set +e
+ cp /var/named/updates.spamassassin.org.d/3.3.3 
/tmp/sa-mkupdate-6153/dns-backup/.
cp: cannot stat `/var/named/updates.spamassassin.org.d/3.3.3': 
Permission denied

+ set -e
+ echo 3.3.3 TXT '"1588859"'
/home/updatesd/svn/mkupdates-with-scores/mkupdate-with-scores: line 125: 
/var/named/updates.spamassassin.org.d/3.3.3.mkupdate-with-scores.new: 
Permission denied
+ mv 
/var/named/updates.spamassassin.org.d/3.3.3.mkupdate-with-scores.new 
/var/named/updates.spamassassin.org.d/3.3.3
mv: accessing `/var/named/updates.spamassassin.org.d/3.3.3': Permission 
denied

+ return 0
+ UPDATED_VERSIONS=4
+ echo 'VERSIONS UPDATE PASSED ON: 4'
VERSIONS UPDATE PASSED ON: 4
+ '[' 4 -gt 0 ']'
+ EXIT=0
+ copy_update_paranoid /tmp/sa-mkupdate-6153/1588859.tar.gz 
/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz

+ SRC=/tmp/sa-mkupdate-6153/1588859.tar.gz
+ DST=/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
+ set +e
+ cp /tmp/sa-mkupdate-6153/1588859.tar.gz 
/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
cp: cannot create regular file 
`/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz': 
Permission denied
+ diff -u /tmp/sa-mkupdate-6153/1588859.tar.gz 
/var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz
diff: /var/www/buildbot.spamassassin.org/updatestage/1588859.tar.gz: No 
such file or directory

+ '[' 2 -ne 0 ']'
+ set -e
+ return 1
+ EXIT=5
+ '[' 5 -gt 0 ']'
+ (( I=0 ))
+ (( I<=3 ))
+ revert_dns_record 3.3.0
+ SA_VERSION=3.3.0
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.0
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.0 
/var/named/updates.spamassassin.org.d/3.3.0
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.0': Permission 
denied

+ set -e
+ (( I++  ))
+ (( I<=3 ))
+ revert_dns_record 3.3.1
+ SA_VERSION=3.3.1
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.1
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.1 
/var/named/updates.spamassassin.org.d/3.3.1
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.1': Permission 
denied

+ set -e
+ (( I++  ))
+ (( I<=3 ))
+ revert_dns_record 3.3.2
+ SA_VERSION=3.3.2
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.2
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.2 
/var/named/updates.spamassassin.org.d/3.3.2
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.2': Permission 
denied

+ set -e
+ (( I++  ))
+ (( I<=3 ))
+ revert_dns_record 3.3.3
+ SA_VERSION=3.3.3
+ '[' 0 -eq 1 -a 0 -eq 0 ']'
+ DNSFILE=/var/named/updates.spamassassin.org.d/3.3.3
+ set +e
+ cp /tmp/sa-mkupdate-6153/dns-backup/3.3.3 
/var/named/updates.spamassassin.org.d/3.3.3
cp: accessing `/var/named/updates.spamassassin.org.d/3.3.3': Permission 
denied

+ set -e
+ (( I++  ))
+ (( I<=3 ))
+ exit 5
Exit Status 5 is not zero for mkupdate-with-scores


--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: Next priority to get running on sa-vm1.apache.org

[Kevin A. McGrail]

On 5/7/2017 3:43 PM, Dave Jones wrote:
Do you know what the long-term plan is for the /x1 mount?  Was it 
setup as a temporary place to restore old backups?


I am trying to organize everything under /usr/local/spamassassin and 
the root FS doesn't have enough space.  If the plan was to eventually 
unmount and give back the /x1 storage, we can do that. If we are going 
able to keep it, then I would like to remount it under 
/usr/local/spamassassin if that is OK.


root@sa-vm1:/usr/local/spamassassin# df -H
Filesystem  Size  Used Avail Use% Mounted on
/dev/sda134G  8.9G   24G  28% /
/dev/sdb1   1.2T  426G  678G  39% /x1 


If we need the space, use it and remount how you would like!

Regards,

KAM


--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Re: Fwd: Re: Copy of the current SA Zone?

[Kevin A. McGrail]

On 5/2/2017 9:31 AM, Dave Jones wrote:


On 5/2/17 8:20 AM, Kevin A. McGrail wrote:

On 5/2/2017 9:14 AM, Dave Jones wrote:
My plan is to setup a script on sa-vm1.apache.org that would run 
daily and email if there are record differences since we don't have 
control of the public DNS servers. 


I seem to remember that they might use cron syncs and not respond to 
notifies so expect FPs.


I realize there are some timings involved based on when a change is 
made but we shouldn't get FPs two days in a row.  They should never 
get more than 24 hours behind.  If they are running their cron jobs 
every 15 minutes per your notes/previous email, then the average is 
only going to be 7-8 minutes behind.  The odds of an FP in this case 
should be pretty low.  If this becomes an issue, I can add logic to 
the script to detect different serials and do a delayed retry an hour 
later or something. 
That might not be accurate built on an assumption that they have 15 
minute crons.  Did I say 15 minutes?


We publish releases every day so the zone changes every day.  I do not 
know how often they run crons but the lag was not like it is with my 
name servers with slaves/public/hiddens with notify and fairly instant 
updates.


We can work to improve it if Hyperreal and Sonic are game but I'm 
worried you are just going to have a script always saying we are out of 
sync.


--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project

Fwd: Re: Copy of the current SA Zone?

[Kevin A. McGrail]

And the sonic copy




spamassassin.org
Description: Lotus Organizer