Looks like this is already taken care of ? Prashant Sharma
On Mon, Sep 8, 2014 at 4:37 AM, Josh Rosen <rosenvi...@gmail.com> wrote: > Does anyone know why some of the MiMa tests have started failing? > > See > https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19948/consoleFull > for > an example. > > On September 6, 2014 at 12:48:27 PM, Josh Rosen (rosenvi...@gmail.com) > wrote: > > It looks like Jenkins is up and running, but there seems to be a delay in > responding to requests to re-test patches. It seems like Jenkins is > promptly testing new PRs, or new commits as they’re added to existing PRs, > but taking a very long time to respond to requests to re-test PRs. > > I’m going to continue monitoring this today. I’m considering creating my > own fork of the Jenkins pull request builder plugin so that we can add > extra logging in order to diagnose what’s causing this lag. > > - Josh > On September 5, 2014 at 5:49:32 PM, Josh Rosen (rosenvi...@gmail.com) > wrote: > > We have successfully purged Jenkins’ build queue. If you want a PR to be > re-tested, please ask Jenkins again. > > On September 5, 2014 at 5:36:30 PM, shane knapp (skn...@berkeley.edu) > wrote: > > yeah, it was a problem w/the PRB's OAuth key. josh rosen added a new key, > and magique! > > we're about to clear the queue of all builds as most aren't wanted/needed. > > > On Fri, Sep 5, 2014 at 5:33 PM, Nicholas Chammas < > nicholas.cham...@gmail.com > > wrote: > > > Looks like Jenkins is back! > > > > lol The poor guy has like a million builds > > < > https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/ > > > > to catch up on. > > > > > > On Fri, Sep 5, 2014 at 4:15 PM, Nicholas Chammas < > > nicholas.cham...@gmail.com> wrote: > > > >> How's it going? > >> > >> It looks like during the last build > >> < > https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/lastBuild/console > > > >> from about 30 min ago Jenkins was still having trouble fetching from > >> GitHub. It also looks like not all requests for testing are triggering > >> builds. > >> > >> > >> On Fri, Sep 5, 2014 at 1:23 PM, shane knapp <skn...@berkeley.edu> > wrote: > >> > >>> it's looking like everything except the pull request builders are > >>> working. i'm going to be working on getting this resolved today. > >>> > >>> > >>> On Fri, Sep 5, 2014 at 8:18 AM, Nicholas Chammas < > >>> nicholas.cham...@gmail.com> wrote: > >>> > >>>> Hmm, looks like at least some builds > >>>> < > https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19804/consoleFull > > > >>>> are working now, though this last one was from ~5 hours ago. > >>>> > >>>> > >>>> On Fri, Sep 5, 2014 at 1:02 AM, shane knapp <skn...@berkeley.edu> > >>>> wrote: > >>>> > >>>>> yep. that's exactly the behavior i saw earlier, and will be figuring > >>>>> out first thing tomorrow morning. i bet it's an environment issues > on the > >>>>> slaves. > >>>>> > >>>>> > >>>>> On Thu, Sep 4, 2014 at 7:10 PM, Nicholas Chammas < > >>>>> nicholas.cham...@gmail.com> wrote: > >>>>> > >>>>>> Looks like during the last build > >>>>>> < > https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/19797/console > > > >>>>>> Jenkins was unable to execute a git fetch? > >>>>>> > >>>>>> > >>>>>> On Thu, Sep 4, 2014 at 7:58 PM, shane knapp <skn...@berkeley.edu> > >>>>>> wrote: > >>>>>> > >>>>>>> i'm going to restart jenkins and see if that fixes things. > >>>>>>> > >>>>>>> > >>>>>>> On Thu, Sep 4, 2014 at 4:56 PM, shane knapp <skn...@berkeley.edu> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> looking > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, Sep 4, 2014 at 4:21 PM, Nicholas Chammas < > >>>>>>>> nicholas.cham...@gmail.com> wrote: > >>>>>>>> > >>>>>>>>> It appears that our main man is having trouble > >>>>>>>>> < > https://amplab.cs.berkeley.edu/jenkins/view/Pull%20Request%20Builders/job/SparkPullRequestBuilder/ > > > >>>>>>>>> hearing new requests > >>>>>>>>> <https://github.com/apache/spark/pull/2277#issuecomment-54549106 > >. > >>>>>>>>> > >>>>>>>>> Do we need some smelling salts? > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Thu, Sep 4, 2014 at 5:49 PM, shane knapp <skn...@berkeley.edu > > > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> i'd ping the Jenkinsmench... the master was completely offline, > >>>>>>>>>> so any new > >>>>>>>>>> jobs wouldn't have reached it. any jobs that were queued when > >>>>>>>>>> power was > >>>>>>>>>> lost probably started up, but jobs that were running would fail. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Thu, Sep 4, 2014 at 2:45 PM, Nicholas Chammas < > >>>>>>>>>> nicholas.cham...@gmail.com > >>>>>>>>>> > wrote: > >>>>>>>>>> > >>>>>>>>>> > Woohoo! Thanks Shane. > >>>>>>>>>> > > >>>>>>>>>> > Do you know if queued PR builds will automatically be picked > >>>>>>>>>> up? Or do we > >>>>>>>>>> > have to ping the Jenkinmensch manually from each PR? > >>>>>>>>>> > > >>>>>>>>>> > Nick > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > On Thu, Sep 4, 2014 at 5:37 PM, shane knapp < > >>>>>>>>>> skn...@berkeley.edu> wrote: > >>>>>>>>>> > > >>>>>>>>>> >> AND WE'RE UP! > >>>>>>>>>> >> > >>>>>>>>>> >> sorry that this took so long... i'll send out a more detailed > >>>>>>>>>> explanation > >>>>>>>>>> >> of what happened soon. > >>>>>>>>>> >> > >>>>>>>>>> >> now, off to back up jenkins. > >>>>>>>>>> >> > >>>>>>>>>> >> shane > >>>>>>>>>> >> > >>>>>>>>>> >> > >>>>>>>>>> >> On Thu, Sep 4, 2014 at 1:27 PM, shane knapp < > >>>>>>>>>> skn...@berkeley.edu> wrote: > >>>>>>>>>> >> > >>>>>>>>>> >> > it's a faulty power switch on the firewall, which has been > >>>>>>>>>> swapped out. > >>>>>>>>>> >> > we're about to reboot and be good to go. > >>>>>>>>>> >> > > >>>>>>>>>> >> > > >>>>>>>>>> >> > On Thu, Sep 4, 2014 at 1:19 PM, shane knapp < > >>>>>>>>>> skn...@berkeley.edu> > >>>>>>>>>> >> wrote: > >>>>>>>>>> >> > > >>>>>>>>>> >> >> looks like some hardware failed, and we're swapping in a > >>>>>>>>>> replacement. > >>>>>>>>>> >> i > >>>>>>>>>> >> >> don't have more specific information yet -- including > >>>>>>>>>> *what* failed, > >>>>>>>>>> >> as our > >>>>>>>>>> >> >> sysadmin is super busy ATM. the root cause was an > >>>>>>>>>> incorrect circuit > >>>>>>>>>> >> being > >>>>>>>>>> >> >> switched off during building maintenance. > >>>>>>>>>> >> >> > >>>>>>>>>> >> >> on a side note, this incident will be accelerating our > plan > >>>>>>>>>> to move the > >>>>>>>>>> >> >> entire jenkins infrastructure in to a managed datacenter > >>>>>>>>>> environment. > >>>>>>>>>> >> this > >>>>>>>>>> >> >> will be our major push over the next couple of weeks. more > >>>>>>>>>> details > >>>>>>>>>> >> about > >>>>>>>>>> >> >> this, also, as soon as i get them. > >>>>>>>>>> >> >> > >>>>>>>>>> >> >> i'm very sorry about the downtime, we'll get everything up > >>>>>>>>>> and running > >>>>>>>>>> >> >> ASAP. > >>>>>>>>>> >> >> > >>>>>>>>>> >> >> > >>>>>>>>>> >> >> On Thu, Sep 4, 2014 at 12:27 PM, shane knapp < > >>>>>>>>>> skn...@berkeley.edu> > >>>>>>>>>> >> wrote: > >>>>>>>>>> >> >> > >>>>>>>>>> >> >>> looks like a power outage in soda hall. more updates as > >>>>>>>>>> they happen. > >>>>>>>>>> >> >>> > >>>>>>>>>> >> >>> > >>>>>>>>>> >> >>> On Thu, Sep 4, 2014 at 12:25 PM, shane knapp < > >>>>>>>>>> skn...@berkeley.edu> > >>>>>>>>>> >> >>> wrote: > >>>>>>>>>> >> >>> > >>>>>>>>>> >> >>>> i am trying to get things up and running, but it looks > >>>>>>>>>> like either > >>>>>>>>>> >> the > >>>>>>>>>> >> >>>> firewall gateway or jenkins server itself is down. i'll > >>>>>>>>>> update as > >>>>>>>>>> >> soon as > >>>>>>>>>> >> >>>> i know more. > >>>>>>>>>> >> >>>> > >>>>>>>>>> >> >>> > >>>>>>>>>> >> >>> > >>>>>>>>>> >> >> > >>>>>>>>>> >> > > >>>>>>>>>> >> > >>>>>>>>>> > > >>>>>>>>>> > -- > >>>>>>>>>> > You received this message because you are subscribed to the > >>>>>>>>>> Google Groups > >>>>>>>>>> > "amp-infra" group. > >>>>>>>>>> > To unsubscribe from this group and stop receiving emails from > >>>>>>>>>> it, send an > >>>>>>>>>> > email to amp-infra+unsubscr...@googlegroups.com. > >>>>>>>>>> > For more options, visit https://groups.google.com/d/optout. > >>>>>>>>>> > > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > > >
