[Toolserver-l] Result of the general member meeting of WMDE

[DaB.]

Hello all,

I just got back from the general member meeting of Wikimedia Deutschland. As 
you know I requested a decision about the future of the toolserver there. To 
make it short: It doesn't went as well as I hoped. While the request itself 
was accepted, it was changed in some important parts.
The main fear was that WMF could stop to provide us with fresh dumps and/or 
replication in near future, making the toolserver more or less useless. 
Although I learned from a participating WMF-board-member that no such board-
decision exists. 
My request was changed in the following way: The WMF has to tell WMDE within 6 
months how Wikilabs can replace the toolserver in the promised complete way. 
If the answer is not satisfying, WMDE will develop a "Governance-Model" to 
ensure the continuation of the toolserver. Different groups are invited into 
this "Governance-Model" and it should be done until the end of 2013.
That sounds good on the first view, but there are 2 loop-holes: Nobody defined 
what "complete" or "satisfying" is. In my eyes Wikilabs can not replace the 
toolserver complete (in the way that all tools can move to there) and so the 
answer can only be unsatisfying, but that's just a question of definition I 
guess.
A second change was that the investment for the toolserver will be restricted 
to the "necessary". While that is of course a matter of definition again I'm 
sure that means "no new hardware if it is possible in any way".

To summarize this: In the best case we have to wait for 6 months until WMDE 
officially learns that Wikilabs can not replace us, than wait for another 6 
months until they will create their "Governance-Model" and in 2014 we get new 
hardware.
In worst case we wait for 6 months and than WMDE and WMF agree that everything 
is ok and we will never get any new hardware and somewhen the TS will shut 
down (of course with the remaining tools that can not migrated to Wikilabs).
I can not imagine ways between both cases, but I'm sure they exists. In any 
way we will get no (or nearly no) new hardware in 2013 – so we have to life 
with that.

A good news is that the toolserver will get 3 new database-servers soon.

I have not decided yet if I will remain as root under this circumstances for 
2013 – I will tell you my decision until next Sunday.

For now I will head to bed because I'm exhausted and disappointed. See you 
tomorrow.

Sincerely,
DaB.

 

-- 
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885


signature.asc
Description: This is a digitally signed message part.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Anyone else having TS trouble?

[Marlen Caemmerer]

Hello,

yes it seems the same error happened that occured on Friday.
ngingx seems to be running nuts / the memory of the host explodes and the 
system boots.
I ll look into the root cause of this tomorrow but I can already tell you the 
system sort of healed itself - but there is something broken
.

Cheers
nosy


___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] SGE queue waiting forever?

[Marlen Caemmerer]

Hello,

nice to hear. 
I had a look from all sides but it seemed the SGE master thought the queues on the hosts were full.

This morning when I looked I saw only willow doing some jobs - ortelius still 
having this strange state.
I waited for Merl to advise me probably reconfiguring but it was too early this 
morning and I simply deleted some (5 i think)
jobs from the queues that were issued on 9th Oct when user-store failed.

I felt the users might probably not wait for this job until now anyway and 
hoped the queues would regenerate as they were modified.
This seems to have solved the problem to my luck ;) as in the logs it seems the 
jobs were running fine then.

Cheers
nosy



On Sun, 25 Nov 2012, Dr. Trigon wrote:


Date: Sun, 25 Nov 2012 11:33:19
From: Dr. Trigon 
Reply-To: Wikimedia Toolserver 
To: toolserver-l@lists.wikimedia.org
Subject: Re: [Toolserver-l] SGE queue waiting forever?

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Today it seems to be working and fully functional again... Nice job! ;)

Thanks to all involved here!

Greetings and have a nice weekend
DrTrigon


On 24.11.2012 22:10, Platonides wrote:

On 24/11/12 21:38, Dr. Trigon wrote:

@All: If you are working on big files please copy them to local
 temp first (on sge $TMP contains an individual temp dir for
the job). E.g. piping big files to other slow programs causes
much nfs load because data must be read in small packages which
cause high load on servers. That's why sge cannot schedule new
jobs on nightshade since days.


What is a big file? Is it ok if the file is in user-home?

Thanks and greetings DrTrigon


/home is also mounted with nfs

Although it's strange that reading from big files overloads the
servers. stdio or the equivalent functionality in the language
they are made should be making it work in blocks.

Looking at willow mounts, /shared and /home are mounted with nfsv3
over udp. But /mnt/user-store and /install don't show it, so they
are probably using nfsv4 over tcp. Is that intended?



___ Toolserver-l
mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting
guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette



-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCx8+8ACgkQAXWvBxzBrDBSyQCfc7mOdoj45Phyx0p+9Be5sm99
tdcAn0m3hTWswEuvfBGAIBlsmMW9uhNO
=+rBS
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette




___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Anyone else having TS trouble?

[Magnus Manske]

Works for me now.


On Sun, Nov 25, 2012 at 6:19 PM, Krinkle  wrote:

> On Nov 25, 2012, at 5:06 PM, Platonides  wrote:
>
> On 25/11/12 16:42, Magnus Manske wrote:
>
> Started a few minutes ago. getting alternating errors on reloading a
> page; 500 (on a static HTML page, not CGI, mind you!), "This webpage is
> not available: The connection to toolserver.org 
> was interrupted.", 404.
>
>
> The same thing happpened here.
> The servers didn't allow logins, and accessing the web pages return
> 404s. Apparently ldap was down.
> It recovered now. I don't know if it there was a human involved, if it
> solved itself or if it was turnera taking over damiana automatically.
>
>
> Seems fine here:
>
> https://toolserver.org/~krinkle/mwSnapshots/#!/mediawiki-core/master
>
> Maybe HTTP or Geo related? Is anyone still having this problem?
>
> -- Krinkle
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Anyone else having TS trouble?

[Krinkle]

On Nov 25, 2012, at 5:06 PM, Platonides  wrote:

> On 25/11/12 16:42, Magnus Manske wrote:
>> Started a few minutes ago. getting alternating errors on reloading a
>> page; 500 (on a static HTML page, not CGI, mind you!), "This webpage is
>> not available: The connection to toolserver.org 
>> was interrupted.", 404.
> 
> The same thing happpened here.
> The servers didn't allow logins, and accessing the web pages return
> 404s. Apparently ldap was down.
> It recovered now. I don't know if it there was a human involved, if it
> solved itself or if it was turnera taking over damiana automatically.
> 

Seems fine here:

https://toolserver.org/~krinkle/mwSnapshots/#!/mediawiki-core/master

Maybe HTTP or Geo related? Is anyone still having this problem?

-- Krinkle

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Anyone else having TS trouble?

[Platonides]

On 25/11/12 16:42, Magnus Manske wrote:
> Started a few minutes ago. getting alternating errors on reloading a
> page; 500 (on a static HTML page, not CGI, mind you!), "This webpage is
> not available: The connection to toolserver.org 
> was interrupted.", 404.

The same thing happpened here.
The servers didn't allow logins, and accessing the web pages return
404s. Apparently ldap was down.
It recovered now. I don't know if it there was a human involved, if it
solved itself or if it was turnera taking over damiana automatically.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Anyone else having TS trouble?

[Christian Thiele]

Am 25.11.2012, 16:42 Uhr, schrieb Magnus Manske  
:


Started a few minutes ago. getting alternating errors on reloading a  
page;

500 (on a static HTML page, not CGI, mind you!), "This webpage is not
available: The connection to toolserver.org was interrupted.", 404.


same here...

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Anyone else having TS trouble?

[Magnus Manske]

Started a few minutes ago. getting alternating errors on reloading a page;
500 (on a static HTML page, not CGI, mind you!), "This webpage is not
available: The connection to toolserver.org was interrupted.", 404.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] SGE queue waiting forever?

[Dr. Trigon]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Today it seems to be working and fully functional again... Nice job! ;)

Thanks to all involved here!

Greetings and have a nice weekend
DrTrigon


On 24.11.2012 22:10, Platonides wrote:
> On 24/11/12 21:38, Dr. Trigon wrote:
>>> @All: If you are working on big files please copy them to local
>>>  temp first (on sge $TMP contains an individual temp dir for
>>> the job). E.g. piping big files to other slow programs causes
>>> much nfs load because data must be read in small packages which
>>> cause high load on servers. That's why sge cannot schedule new
>>> jobs on nightshade since days.
>> 
>> What is a big file? Is it ok if the file is in user-home?
>> 
>> Thanks and greetings DrTrigon
> 
> /home is also mounted with nfs
> 
> Although it's strange that reading from big files overloads the 
> servers. stdio or the equivalent functionality in the language
> they are made should be making it work in blocks.
> 
> Looking at willow mounts, /shared and /home are mounted with nfsv3 
> over udp. But /mnt/user-store and /install don't show it, so they
> are probably using nfsv4 over tcp. Is that intended?
> 
> 
> 
> ___ Toolserver-l
> mailing list (Toolserver-l@lists.wikimedia.org) 
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting
> guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
> 

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlCx8+8ACgkQAXWvBxzBrDBSyQCfc7mOdoj45Phyx0p+9Be5sm99
tdcAn0m3hTWswEuvfBGAIBlsmMW9uhNO
=+rBS
-END PGP SIGNATURE-

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette