Re: amplab jenkins is down

2014-09-05 Thread shane knapp
/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19804/consoleFull are working now, though this last one was from ~5 hours ago. On Fri, Sep 5, 2014 at 1:02 AM, shane knapp skn...@berkeley.edu wrote: yep. that's exactly the behavior i saw earlier, and will be figuring out first thing

amplab jenkins is down

2014-09-04 Thread shane knapp
i am trying to get things up and running, but it looks like either the firewall gateway or jenkins server itself is down. i'll update as soon as i know more.

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
looks like a power outage in soda hall. more updates as they happen. On Thu, Sep 4, 2014 at 12:25 PM, shane knapp skn...@berkeley.edu wrote: i am trying to get things up and running, but it looks like either the firewall gateway or jenkins server itself is down. i'll update as soon as i

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
ASAP. On Thu, Sep 4, 2014 at 12:27 PM, shane knapp skn...@berkeley.edu wrote: looks like a power outage in soda hall. more updates as they happen. On Thu, Sep 4, 2014 at 12:25 PM, shane knapp skn...@berkeley.edu wrote: i am trying to get things up and running, but it looks like either

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
it's a faulty power switch on the firewall, which has been swapped out. we're about to reboot and be good to go. On Thu, Sep 4, 2014 at 1:19 PM, shane knapp skn...@berkeley.edu wrote: looks like some hardware failed, and we're swapping in a replacement. i don't have more specific

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
AND WE'RE UP! sorry that this took so long... i'll send out a more detailed explanation of what happened soon. now, off to back up jenkins. shane On Thu, Sep 4, 2014 at 1:27 PM, shane knapp skn...@berkeley.edu wrote: it's a faulty power switch on the firewall, which has been swapped out

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
/pull/2277#issuecomment-54549106. Do we need some smelling salts? On Thu, Sep 4, 2014 at 5:49 PM, shane knapp skn...@berkeley.edu wrote: i'd ping the Jenkinsmench... the master was completely offline, so any new jobs wouldn't have reached it. any jobs that were queued when power was lost

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
i'm going to restart jenkins and see if that fixes things. On Thu, Sep 4, 2014 at 4:56 PM, shane knapp skn...@berkeley.edu wrote: looking On Thu, Sep 4, 2014 at 4:21 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: It appears that our main man is having trouble https

Re: amplab jenkins is down

2014-09-04 Thread shane knapp
://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19797/console Jenkins was unable to execute a git fetch? On Thu, Sep 4, 2014 at 7:58 PM, shane knapp skn...@berkeley.edu wrote: i'm going to restart jenkins and see if that fixes things. On Thu, Sep 4, 2014 at 4:56

hey spark developers! intro from shane knapp, devops engineer @ AMPLab

2014-09-02 Thread shane knapp
so, i had a meeting w/the databricks guys on friday and they recommended i send an email out to the list to say 'hi' and give you guys a quick intro. :) hi! i'm shane knapp, the new AMPLab devops engineer, and will be spending time getting the jenkins build infrastructure up to production

Re: quick jenkins restart

2014-09-02 Thread shane knapp
and we're back and building! On Tue, Sep 2, 2014 at 5:07 PM, shane knapp skn...@berkeley.edu wrote: since our queue is really short, i'm waiting for a couple of builds to finish and will be restarting jenkins to install/update some plugins. the github pull request builder looks like it has

Re: emergency jenkins restart, aug 29th, 730am-9am PDT -- plus a postmortem

2014-08-29 Thread shane knapp
reminder: this is happening right now. jenkins is currently in quiet mode, and in ~30 minutes, will be briefly going down. On Thu, Aug 28, 2014 at 1:03 PM, shane knapp skn...@berkeley.edu wrote: as with all software upgrades, sometimes things don't always work as expected. a recent

Re: emergency jenkins restart, aug 29th, 730am-9am PDT -- plus a postmortem

2014-08-29 Thread shane knapp
this is done. On Fri, Aug 29, 2014 at 7:32 AM, shane knapp skn...@berkeley.edu wrote: reminder: this is happening right now. jenkins is currently in quiet mode, and in ~30 minutes, will be briefly going down. On Thu, Aug 28, 2014 at 1:03 PM, shane knapp skn...@berkeley.edu wrote

new jenkins plugin installed and ready for use

2014-08-29 Thread shane knapp
i have always found the 'Rebuild' plugin super useful: https://wiki.jenkins-ci.org/display/JENKINS/Rebuild+Plugin this is installed and enables. enjoy! shane

Re: jenkins maintenance/downtime, aug 28th, 730am-9am PDT

2014-08-28 Thread shane knapp
jenkins is now coming down. On Thu, Aug 28, 2014 at 7:19 AM, shane knapp skn...@berkeley.edu wrote: reminder: this is starting in 10 minutes On Wed, Aug 27, 2014 at 4:13 PM, shane knapp skn...@berkeley.edu wrote: tomorrow morning i will be upgrading jenkins to the latest/greatest (1.577

Re: jenkins maintenance/downtime, aug 28th, 730am-9am PDT

2014-08-28 Thread shane knapp
! :) On Thu, Aug 28, 2014 at 7:46 AM, shane knapp skn...@berkeley.edu wrote: jenkins is now coming down. On Thu, Aug 28, 2014 at 7:19 AM, shane knapp skn...@berkeley.edu wrote: reminder: this is starting in 10 minutes On Wed, Aug 27, 2014 at 4:13 PM, shane knapp skn...@berkeley.edu

Re: jenkins maintenance/downtime, aug 28th, 730am-9am PDT

2014-08-28 Thread shane knapp
this one job is blocking the jenkins restart: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19406/ i'm about to kill it so that i can get this done. i'll restart the job after jenkins is back up. On Thu, Aug 28, 2014 at 7:51 AM, shane knapp skn...@berkeley.edu wrote

Re: jenkins maintenance/downtime, aug 28th, 730am-9am PDT

2014-08-28 Thread shane knapp
all clear: jenkins and all plugins have been updated! On Thu, Aug 28, 2014 at 7:51 AM, shane knapp skn...@berkeley.edu wrote: jenkins is upgraded, but a few jobs sneaked in before i could do the plugin updates. i've put jenkins in quiet mode again, and once the spark builds finish, i'll

Re: jenkins maintenance/downtime, aug 28th, 730am-9am PDT

2014-08-28 Thread shane knapp
this, Shane. On Thursday, August 28, 2014, shane knapp skn...@berkeley.edu wrote: all clear: jenkins and all plugins have been updated! On Thu, Aug 28, 2014 at 7:51 AM, shane knapp skn...@berkeley.edu wrote: jenkins is upgraded, but a few jobs sneaked in before i could do the plugin updates

emergency jenkins restart, aug 29th, 730am-9am PDT -- plus a postmortem

2014-08-28 Thread shane knapp
as with all software upgrades, sometimes things don't always work as expected. a recent change to stapler[1], to verbosely report NotExportableExceptions[2] is spamming our jenkins log file with stack traces, which is growing rather quickly (1.2G since 9am). this has been reported to the jenkins

<    3   4   5   6   7   8