On Sun, 16 Jul 2017 13:10:07 -0500 Derek Foreman <der...@osg.samsung.com> said:
> That was me, but I've been monitoring from outside - I have no idea if > it's the colo provider doing something, or some cron job at 5:00-5:10AM > CST, or ??? i've never been unable to get into e5.e.org (which is the host) at any time unless it's being rebooted at the time. which would rule out the hosting. but this gels with my "when it goes down... it's often towards the end of my day" ... 7pmish. some days i'm off to korean lessons from 7 to 8 so it's working before lesson, broken when i get home about 8:30... so this gels with what i've seen.. but the host (e5) was alive. just rebooting it fixed things. i do know beber seems to have rebuilt the kernel recently and is running it. Jul 11 16:11:10 specifically. kvm is now a module.... so next time the machine is being "dead" (not just ssh denying because of proxy command config or something) ... i can shut down qemu vm's and rmmod/modprobe and see ... but the last few times i ssh'd into e5v1 and git/phab/www etc. and they were responsive on the cmdline. so things seem to have changed how they break... but now we talk about this.... ctron.daily runs at 3:05am... that's 5:05am CST. (e.org is in PDT). and... at 3am it does logrotate.... what i thought long ago was that this was some kind of timeout... as if it were something logging every command run or something ands it's timing out trying to send logs... i think we have something here... the smell is strong. > On 2017-07-16 12:08 PM, Jonathan Aquilina wrote: > > Not sure but someone mentioned at a particular time that the colo provider > > does something with the network which I'm not sure of its related or not. > > > > Sent from my iPhone > > > >> On 16 Jul 2017, at 18:27, Andrew Williams <a...@andywilliams.me> wrote: > >> > >> Ok, got seems to be just fine today. > >> Anyone know what happened? > >> > >> Thanks, > >> Andy > >>> On Sun, 16 Jul 2017 at 00:32, Andrew Williams <a...@andywilliams.me> > >>> wrote: Hi, > >>> > >>> After an hour of trying to get through to our git server I'm now giving > >>> up for the night. I understand that there is an unknown problem that > >>> needs to be gotten to the bottom of but I don't understand why those who > >>> were able to help no longer have access. > >>> > >>> Can someone in the know please communicate what is going on? I know that > >>> we are getting our hosting provided kindly for free but There are so many > >>> free hosting solutions for open source projects I don't get it - we were > >>> afraid of the stability of "the new sourceforge" but GitHub, despite high > >>> criticism for its downtime, has been unavailable for barely 2 days over > >>> the last 2 years... > >>> > >>> Andy > >>>> On Sat, 15 Jul 2017 at 22:41, jaquilina <jaquil...@eagleeyet.net> wrote: > >>>> All I know is that beber adding my ssh key to the system and reencrypted > >>>> the credentials store which could be the root cause of these issues with > >>>> git over ssh > >>>> > >>>> > >>>> > >>>> Sent from Samsung tablet. > >>>> -------- Original message -------- > >>>> From: Andrew Williams <a...@andywilliams.me> > >>>> Date: 15/07/2017 22:59 (GMT+01:00) > >>>> To: Jonathan Aquilina <jaquil...@eagleeyet.net>, Enlightenment developer > >>>> list <enlightenment-devel@lists.sourceforge.net> Subject: Re: [E-devel] > >>>> Server down > >>>> > >>>> Correction, for git I'm getting either connection closed or permission > >>>> denied... every 5th or so connection makes it through. > >>>>> On Sat, 15 Jul 2017 at 20:46, Andrew Williams <a...@andywilliams.me> > >>>>> wrote: Hi, > >>>>> > >>>>> Thanks to whoever prodded it. Now git and web are working again. > >>>>> I use git+ssh and it is working OK (on and off actually) but I can't > >>>>> get in through ssh any more. Bu5hm4n and others have noted the same > >>>>> thing. Have permissions on SSH been altered? > >>>>> > >>>>> Thanks, > >>>>> Andy > >>>>> > >>>>>> On Sat, 15 Jul 2017 at 15:06 Jonathan Aquilina > >>>>>> <jaquil...@eagleeyet.net> wrote: I think that it might be worth > >>>>>> contacting the hosting provider as I am wondering if they are doing > >>>>>> something with their network hardware or maybe something that is > >>>>>> causing our server to really not be happy. > >>>>>> > >>>>>> --- > >>>>>> Regards, > >>>>>> > >>>>>> Jonathan Aquilina > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On 2017-07-15 15:59, Derek Foreman wrote: > >>>>>>> > >>>>>> > >>>>>>> Somewhat related, I've been monitoring the server with "smokeping" > >>>>>>> for a few weeks now, and the majority of the blackouts start between > >>>>>>> 5AM and 5:10AM north american central time (10ish UTC I guess?) > >>>>>>> > >>>>>>> Sometimes they clear in under an hour (I don't know if this is an > >>>>>>> automatic recovery or manual intervention). > >>>>>>> > >>>>>>> During the blackouts generally the git server is partially responsive > >>>>>>> and the web server is completely borked. > >>>>>>> > >>>>>>> Today's blackout started between 5:00AM and 5:10AM CST. Currently > >>>>>>> the git server is semi responsive and the web server is hosed. > >>>>>>> > >>>>>>> I don't know if that gives any useful input into tracking down what's > >>>>>>> knocking over the house of cards (or if knowing what's knocking it > >>>>>>> over is even useful), but there ya go. :D > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Derek > >>>>>>> > >>>>>>> > >>>>>>>> On 2017-07-15 05:26 AM, Andrew Williams wrote: > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> That's the server keeled over again. I would fix it but today it's > >>>>>>>> rejecting my login. > >>>>>>>> > >>>>>>>> Can someone please help, I was in the middle of a bunch of work on > >>>>>>>> docuwiki that I would like to complete? > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Andy > >>>>>>> > >>>>>>> > >>>>>> > >>>>>>> ------------------------------------------------------------------------------ > >>>>>>> Check out the vibrant tech community on one of the world's most > >>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot > >>>>>>> _______________________________________________ > >>>>>>> enlightenment-devel mailing list > >>>>>>> enlightenment-devel@lists.sourceforge.net > >>>>>>> https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > >>>>> > >>>>> -- > >>>>> http://andywilliams.me > >>>>> http://ajwillia.ms > >>>> > >>>> -- > >>>> http://andywilliams.me > >>>> http://ajwillia.ms > >>> > >>> -- > >>> http://andywilliams.me > >>> http://ajwillia.ms > >> > >> -- > >> http://andywilliams.me > >> http://ajwillia.ms > > ------------------------------------------------------------------------------ > > Check out the vibrant tech community on one of the world's most > > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > > _______________________________________________ > > enlightenment-devel mailing list > > enlightenment-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > > > > > ------------------------------------------------------------------------------ > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > _______________________________________________ > enlightenment-devel mailing list > enlightenment-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/enlightenment-devel > -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- The Rasterman (Carsten Haitzler) ras...@rasterman.com ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
