Re: [Toolserver-l] Can't log into toolserver
Yes, this error is systematic in the last few weeks, see archives. Nemo ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
On 04/10/2013 07:51 PM, Tim Landscheidt wrote: > Does Icinga have graphs? I wasn't sure you were talking to me. :-) Icinga have graphs, but they're uptime-related. The place with the pretty graphics is ganglia: http://ganglia.wmflabs.org/latest/?c=tools But there is no queued/running job graph right now. It's a very good idea though, and I'll add the matching instrumentation. -- Marc ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
(anonymous) wrote: > [...] > If you are using sge you have not really care about. If you > can use the hole cluster (linux and solaris) we mostly have > enough capacity. It is only important that you can specify > which resources (memory, runtime) you need. > [...] BTW, is the "Queue State" on http://munin.toolserver.org/Miscellaneous/turnera/index.html#sge the number of queued jobs? Mark, is there something Munin- like on Labs? Does Icinga have graphs? Tim ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
Den 10-04-2013 21:35, Merlissimo skrev: If you are using sge you have not really care about. If you can use the hole cluster (linux and solaris) we mostly have enough capacity. It is only important that you can specify which resources (memory, runtime) you need. If you need user database access on s3 you simple add -l sql-s3-user=1. If you rise the number of db-resources replag must be lower to get your job scheduled (e.g. -l sql-s3-user=3 currently gets only scheduled if replag is below 1 hour). deadline option is not available on toolserver. -p mainly changes to priority compared to other jobs of yourself. For the global scheduling order job waiting time and used server resources by your user account in the last hours is more important. Webserver requests which are also causing much database queries are high at 14-23 UTC workdays. Most sge jobs are submittet between 0-3 UTC. Thank you for the explanations to all. They could be used to improve the documentaion for SGE. BTW I can use the whole cluster, as I made this little script to start my compiled C programs: #!/bin/sh ARCH=`uname` PROG=$1 shift /home/byrial/bin/$ARCH/$PROG $@ ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
> > If you rise the number of db-resources replag must be lower to get your > job scheduled (e.g. -l sql-s3-user=3 currently gets only scheduled if > replag is below 1 hour). > That's new to me. Is there some documentation about that somewhere? Petr Onderka [[en:User:Svick]] ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
Am 10.04.2013 20:54, schrieb Marc A. Pelletier: On 04/10/2013 02:41 PM, Byrial Jensen wrote: unless there is some option I can use to tell that. What Tim mean is that, by default, SGE will schedule your job when sufficient resources are effectively available, rather that trying to predict when that will happen. That said, you /can/ specify both a minimal starting time (with -a) and a deadline (with -dl) creating a "window" during which SGE will try to run your job but, in general, it's easier and more reliable to let the gridengine pick the time. If your objective is to have your job run only when few others are trying to use the resources, you can also lower its priority (with -p) so that it will only execute your job when there isn't anything "better" to run. -- Marc If you are using sge you have not really care about. If you can use the hole cluster (linux and solaris) we mostly have enough capacity. It is only important that you can specify which resources (memory, runtime) you need. If you need user database access on s3 you simple add -l sql-s3-user=1. If you rise the number of db-resources replag must be lower to get your job scheduled (e.g. -l sql-s3-user=3 currently gets only scheduled if replag is below 1 hour). deadline option is not available on toolserver. -p mainly changes to priority compared to other jobs of yourself. For the global scheduling order job waiting time and used server resources by your user account in the last hours is more important. Webserver requests which are also causing much database queries are high at 14-23 UTC workdays. Most sge jobs are submittet between 0-3 UTC. Merlissimo ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
On 04/10/2013 02:41 PM, Byrial Jensen wrote: > unless there is some option I can use to tell that. What Tim mean is that, by default, SGE will schedule your job when sufficient resources are effectively available, rather that trying to predict when that will happen. That said, you /can/ specify both a minimal starting time (with -a) and a deadline (with -dl) creating a "window" during which SGE will try to run your job but, in general, it's easier and more reliable to let the gridengine pick the time. If your objective is to have your job run only when few others are trying to use the resources, you can also lower its priority (with -p) so that it will only execute your job when there isn't anything "better" to run. -- Marc ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
Den 10-04-2013 20:06, Tim Landscheidt skrev: Byrial Jensen wrote: I am planning to make some maintenace reports for the Danish Wikipedia at regular intervals, like once a week or once every few days, but the exact time to run the programs doesn't really matter. So what time of the day is it best to run such programs? And is there a way to tell SGE that it may choose the most convenient time to start to job? It will do that by itself, in fact, it is its whole pur- pose :-). Well, no. SGE cannot know if I want the job run as early as possible or if I can happily wait for several hours for a less busy part of the day, unless there is some option I can use to tell that. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] When is the best time of day to run programs?
Byrial Jensen wrote: > I am planning to make some maintenace reports for the Danish > Wikipedia at regular intervals, like once a week or once > every few days, but the exact time to run the programs > doesn't really matter. > So what time of the day is it best to run such programs? > And is there a way to tell SGE that it may choose the most > convenient time to start to job? It will do that by itself, in fact, it is its whole pur- pose :-). Regarding time: | timl@yarrow:~$ perl -we 'my $t = rand (24 * 60); printf "%d:%02d\n", $t / 60, $t % 60;' | 15:18 | timl@yarrow:~$ Does that suit you? :-) Tim ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Login and account
I would recommend opening a ticket in JIRA. On Wed, Apr 10, 2013 at 11:17 PM, Dr. Trigon wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hello beloved Admins! > > Today I recognized that my account has expired (what a pitty) so I > tried as usual to login and renew but just get this: > >> $ ssh drtri...@nightshade.toolserver.org Your account has expired; >> please contact your system administrator Connection closed by >> 185.15.59.201 > > ...so now what to do? ;) > > Thanks and Greetings! > DrTrigon > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.13 (GNU/Linux) > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iEYEARECAAYFAlFlZnMACgkQAXWvBxzBrDCYlwCgox8MIi8m64RiiO7tAQ7QGEA3 > HXUAoIHEkTgYbKxTVzJLcevYgYvFJCZI > =H/od > -END PGP SIGNATURE- > > ___ > Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) > https://lists.wikimedia.org/mailman/listinfo/toolserver-l > Posting guidelines for this list: > https://wiki.toolserver.org/view/Mailing_list_etiquette ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] mayapple config & cron jobs
Was this issue filed in JIRA?
On Wed, Apr 10, 2013 at 11:18 PM, Dr. Trigon wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> PING! Any news here? Why was this ignored?
>
> Greetings
> DrTrigon
>
>
> On 30.03.2013 14:37, Dr. Trigon wrote:
>> Hello everybody! Hello nosy!
>>
>> Do you remember the issue below...? :)
>>
>>
>> On 05.03.2013 13:03, Marlen Caemmerer wrote:
Then another issue - I think (again - sorry but yes! ;) we
have some problem with cron job execution. When looking at my
bots (even thought this is not COMPLETELY clear beacuse of
other issue mentioned before) but also at the lower left of [4]
to be precise [5] and compare "Week 6" with "Week 7" till now
("Week 9") then we clearly have some issues. The first part of
"Week 7" may be until mid "Week 8" looks resonable, but the
rest looks "free floating" to me which may point to some load
or other issue... what do you think? Thanks again.
[4]
http://munin.toolserver.org/Login/hawthorn/cron_jobs_sh.html
[5]
http://munin.toolserver.org/Login/hawthorn/cron_jobs_sh-month.png
>>
>> Thanks for pointing to this. Cron runs again.
>>
>>
>> I think it crashed another time...
>>
>> If you compare "Week 10/11" with "Week 12" till now ("Week 13")
>> then we clearly have some issues. The first part of "Week 11" may
>> be until mid "Week 12" looks resonable, but the rest looks "free
>> floating" to me which may point to some load or other issue... what
>> do you think? Thanks again. :))
>>
>> Greetings and all the best!! DrTrigon
>>
>> ___ Toolserver-l
>> mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting
>> guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>
>
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.13 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlFlZrsACgkQAXWvBxzBrDBUgACcDgZFGQexx3gHpWQ0tzJ3XswI
> +UQAnR8SS9eD/4OAgZhSzp9wZ/1xcZGo
> =ifeV
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] mayapple config & cron jobs
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
PING! Any news here? Why was this ignored?
Greetings
DrTrigon
On 30.03.2013 14:37, Dr. Trigon wrote:
> Hello everybody! Hello nosy!
>
> Do you remember the issue below...? :)
>
>
> On 05.03.2013 13:03, Marlen Caemmerer wrote:
>>> Then another issue - I think (again - sorry but yes! ;) we
>>> have some problem with cron job execution. When looking at my
>>> bots (even thought this is not COMPLETELY clear beacuse of
>>> other issue mentioned before) but also at the lower left of [4]
>>> to be precise [5] and compare "Week 6" with "Week 7" till now
>>> ("Week 9") then we clearly have some issues. The first part of
>>> "Week 7" may be until mid "Week 8" looks resonable, but the
>>> rest looks "free floating" to me which may point to some load
>>> or other issue... what do you think? Thanks again.
>>>
>>> [4]
>>> http://munin.toolserver.org/Login/hawthorn/cron_jobs_sh.html
>>> [5]
>>> http://munin.toolserver.org/Login/hawthorn/cron_jobs_sh-month.png
>>>
>>>
>
>>>
>>>
> Thanks for pointing to this. Cron runs again.
>
>
> I think it crashed another time...
>
> If you compare "Week 10/11" with "Week 12" till now ("Week 13")
> then we clearly have some issues. The first part of "Week 11" may
> be until mid "Week 12" looks resonable, but the rest looks "free
> floating" to me which may point to some load or other issue... what
> do you think? Thanks again. :))
>
> Greetings and all the best!! DrTrigon
>
> ___ Toolserver-l
> mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting
> guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.13 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlFlZrsACgkQAXWvBxzBrDBUgACcDgZFGQexx3gHpWQ0tzJ3XswI
+UQAnR8SS9eD/4OAgZhSzp9wZ/1xcZGo
=ifeV
-END PGP SIGNATURE-
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list:
https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Login and account
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello beloved Admins! Today I recognized that my account has expired (what a pitty) so I tried as usual to login and renew but just get this: > $ ssh drtri...@nightshade.toolserver.org Your account has expired; > please contact your system administrator Connection closed by > 185.15.59.201 ...so now what to do? ;) Thanks and Greetings! DrTrigon -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlFlZnMACgkQAXWvBxzBrDCYlwCgox8MIi8m64RiiO7tAQ7QGEA3 HXUAoIHEkTgYbKxTVzJLcevYgYvFJCZI =H/od -END PGP SIGNATURE- ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Can't log into toolserver
On Wed, 10 Apr 2013 12:04:37 +0200 Silke Meyer wrote: > As you sent this, someone here in the office reported the same problem. > Cheers, > Silke .. Last login: Wed Apr 10 10:08:35 2013 from xyz quota: error while getting quota from ha-nfs.esi:/global/misc/shared for #8180 (id 8180): Connection refused quota: error while getting quota from ha-nfs.esi:/global/home for jkroll (id 8180): Connection refused jkroll@nightshade:~$ quota quota: error while getting quota from ha-nfs.esi:/global/misc/shared for #8180 (id 8180): Connection refused ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] Can't log into toolserver
As you sent this, someone here in the office reported the same problem. Cheers, Silke On 10.04.2013 12:01, Magnus Manske wrote: > Keeps asking for password, but doesn't accept it. login.toolserver.org, > also ortelius, probably more. > > > > ___ > Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) > https://lists.wikimedia.org/mailman/listinfo/toolserver-l > Posting guidelines for this list: > https://wiki.toolserver.org/view/Mailing_list_etiquette > -- Silke Meyer Internes IT-Management und Projektmanagement Toolserver Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin Tel. (030) 219 158 260 http://wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] Can't log into toolserver
Keeps asking for password, but doesn't accept it. login.toolserver.org, also ortelius, probably more. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
Re: [Toolserver-l] [Labs-l] A (21) day in the Labs
Hi Tim and all, 2013/4/8 Tim Landscheidt > "Marc A. Pelletier" wrote: > > > [...] > > > The database replication is also well on its way; you can find the > > current roadmap at: > > > https://wikitech.wikimedia.org/wiki/Tool_Labs/Database_plan > > > [...] > > To quote from there: > > | Overview > > | * All public wikis will be replicated to the LabsDB servers, > | with private user data redacted. > > | * First, data will be replicated to a special set of data- > | base servers (PreLabsDBDBS) that use triggers to rewrite > | or remove private data. They will write row based bin- > | logs. Production shards will map 1:1 with mysql in- > | stances, unlike on toolserver where some are combined via > | a custom replication engine. > > | * Triggers will be created with the help of the redactatron > | schema review tool. > > | * The actual labs databases will replicate from the above > | mentioned databases. Users will access data via views > | that only include reviewed tables and columns to ensure > | that unreviewed tables (such as from a new extension) > | aren't exposed without prior review. > > | * Replicated data will be stored on flash storage, while > | each system will have a traditional disk array attached to > | store labs project data. Users will be able to join > | project tables against wiki tables, but only within the > | current shard. > > | * The labs team will integrate these databases with labs, > | automating database creation and access on a per-project > | basis. > > This means that JOINs for example between wikis and Commons > or Wikidata will not be possible. WTF? One of the stated > goals of Tool Labs is "Provide a location for analytics > work", so any changes here should /enhance/ the possibili- > ties the Toolserver offers and not shrink them. This is BTW > one of the top items on the "Needed Toolserver features" > list. > We will see into this and find out what is possible to do until when. You'll hear more about it soon. Cheers, Silke -- Silke Meyer Internes IT-Management und Projektmanagement Toolserver Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin Tel. (030) 219 158 260 http://wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985. ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
[Toolserver-l] When is the best time of day to run programs?
Hi, I am planning to make some maintenace reports for the Danish Wikipedia at regular intervals, like once a week or once every few days, but the exact time to run the programs doesn't really matter. So what time of the day is it best to run such programs? And is there a way to tell SGE that it may choose the most convenient time to start to job? ___ Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette
