Hi,

git's still down, phab's still greeting me with the ridiculously rude/embarrassing message that seems to imply that it's the fault of the person trying to connect that something's wrong. (Gee, a spanking, thanks!)

Our server problems are a continually annoying barrier to work, and the thought process around debugging them seems fairly baffling - even if disabling jenkins makes things stumble along for a while it's surely not a real fix for anything?

Jenkins runs in userspace, userspace problems shouldn't be able to cripple a server. And whatever we may think jenkins is doing to cause a problem can be done by any other userspace program. Other threads seem to blame a mysterious kernel bug nobody else can trigger - do we think Jenkins is aggravating that for us and us alone?

Turning it off appears to provide zero information as to what the real problem is and costs us useful functionality.

At what point do we consider, say, applying for freedesktop.org hosting? I'm sure I'm going to see arguments that they don't provide something we need, however right now we have exactly no functional services. No web, no phab, no git. Our mailing list continues to work because someone else hosts it for us.

At least having a stable git repository is critical to getting work done and allowing users/packagers/whoever to get the software - we can still run less important services ourselves and have them fail without blocking everyone.

Thanks,
Derek

(And could someone *please* change that stupid "SPANK SPANK SPANK" message? Is that really what we want potential users to see during our perpetual server outtages?)

On 31/05/17 06:14 PM, Carsten Haitzler (The Rasterman) wrote:
On Wed, 31 May 2017 21:08:05 +0100 Bertrand Jacquin <bertr...@jacquin.bzh> said:

On 31/05/2017 13:21, Stefan Schmidt wrote:
Hello.

On 05/29/2017 12:41 PM, Stefan Schmidt wrote:
Hello.

Once again I had problems with the E server being over capacity and
returning 503 when accessing phab.

I often heard claims that the load of Jenkins jobs trigger the bug we
see here. To actually see if the load due to Jenkins jobs is related
to this I now disabled all build triggers on Jenkins. That means no
Jenkins jobs are going to run until I enable these again. be more
careful with your commits.

Right now the last builds are being performed and after that we should
have no new ones.

I will go an re-enable them again if we either see the same server
instabilities without Jenkins running (which proves in my book hat
Jenkins is not the culprit) or in something like 2 or 3 weeks even if
we have no server problems. In the later case we will have to see what
to do.

And phab 503 again. Without any Jenkins jobs running at all. The last
one have been running two days ago.

Beber, you still think Jenkins is to blame?

I will leave it disabled for a few more days to see how that goes.

Yes, this is still my main believe as of today due to qemu layer. We
won't be able to take any conclusion before I've been able to reboot the
physical host after Jenkins build were disabled, this will be done by
tomorrow.

Thanks for disabling the job, that will definitely help to understand
what to blame.
I may be wrong of course, but at least we'll know.

everything is down now... no phab. no e.org. no git. i literally can't do
anything. i'm relying on phab for some patch review discussions (gl thread).
:( i'm not touching e5 so you can poke around and look...

Cheers,
Bertrand

--
Bertrand

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
enlightenment-devel mailing list
enlightenment-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel





------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
enlightenment-devel mailing list
enlightenment-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel

Reply via email to