Re: [Toolserver-l] [Labs-l] A proposal for better tool discoverability

[Magnus Manske]

On Wed, Aug 13, 2014 at 5:16 PM, Hay (Husky)  wrote:

> Hey Magnus,
> thanks! WDQ is now added to the directory. The cronjob should run every
> hour.
>
> Having a list of tools as an array is something i've been thinking
> about as well, will probably add that.
>

Thanks, I'll hold off some of my tools then, until this is done.

>
> 'Grouping' tools can be done by simply adding a category. Or do you
> mean something else?
>

I mean, visually separating groups on your wiki page, within the 
tag. Easier to check if I have a tool added there if mine are all in a
"section". But not really a show-stopper :-)

This is great!

Cheers,
Magnus
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Labs-l] A proposal for better tool discoverability

[Magnus Manske]

Also: Can the directory detect arrays? For example, in the "wikidata-todo"
tool, I have many smaller tools. I could make a file for each, but that
seems overkill.

Directory tool could do "object=>single tool", "array=>multiple tools in
one file".

Cheers,
Magnus


On Wed, Aug 13, 2014 at 4:53 PM, Magnus Manske 
wrote:

> Thanks Husky,
>
> I've added WDQ to try it. At what interval does it refresh?
>
> It might be helpful if I could structure (group) tools on the wiki page.
> Does the directory ignore not-well-formatted lines (e.g. headings)?
>
> Cheers,
> Magnus
>
>
>
> On Wed, Aug 13, 2014 at 4:15 PM, Hay (Husky)  wrote:
>
>> Hey everyone,
>> for those who attended Wikimania, i hope you all had a wonderful time!
>>
>> One of the things that i've talked about at Mania with a few people is
>> the current way of 'tool discovery', or rather, the lack thereof.
>> We've seen really cool things done by the community, but i have a
>> feeling many of the tools go by unnoticed, simply because nobody knows
>> that they exist. Maybe they've been send around here or posted on IRC,
>> but after the initial interest many of them are forgotten and lead a
>> lonely life on our virtual servers.
>>
>> I think this is especially problematic for non-community members for
>> whom the tools might be very useful. For example, as a Wikipedian in
>> Residence and GLAM advocate i have sent numerous mails with links to
>> the various GLAM tools made by Magnus, stats.grok.se, etcetera.
>>
>> But, you might say, we run all these knowledge sites called wikis
>> right? And there are some pages there that list tools, don't they?
>> You're right, the community has put some efforts in a few pages (1).
>> However, IMHO, a wiki doesn't lend itself to easy tool discovery:
>>
>> * The format on how to describe a tool is unclear, thereby leading to
>> both very long and very short descriptions.
>> * It's not possible to easily search for tools in a certain category
>> (e.g. 'GLAM' or 'editcount').
>> * The categorization of tools leads to much clicking around,
>> distracting and confusing users.
>> * There's no way to sort results, for example by the number of people
>> using the tool.
>>
>> The biggest problem of all is the disconnection between the actual
>> code of the tool itself (hosted in a Git repo), and the description
>> advertising it. People update their tool to do something different (or
>> deprecate it), but they forget the documentation on the wiki, don't
>> remove it when the tool doesn't work anymore, etcetera.
>>
>> I believe we can, and should do better.
>>
>> THE PROPOSAL
>>
>> My proposal is to use the same mechanisms that already exist for
>> package managers such as NPM (2) or 'app stores', such as the Chrome
>> store (3). Basically this involes adding a small JSON file to your
>> project, including a few key properties such as 'title' and
>> 'description'. These files get indexed, and an easy to use frontend to
>> search to all the tools is provided to end users. You could imagine a
>> 'toolinfo.json' file to look something like this:
>>
>> {
>> "name" : "WikiDataQuery",
>> "description" : "An API for Wikidata items and properties.",
>> "url" : "http://wdq.wmflabs.org/";,
>> "keywords" : "wikidata, api, query",
>> "author" : "Magnus Manske"
>> }
>>
>> If you have a web-hosted tool, simply stick it in the root of your
>> tools directory so that it's reachable by the crawler. Whenever your
>> tool data changes, just update the file and the directory will
>> automatically update the directory site.
>>
>> The link to your toolinfo.json could be added to a Wiki page so that
>> it's easy to remove your tool from the directory or change the URL to
>> the JSON file.
>>
>> I'm a firm believer in putting code where your mouth is, so i've
>> hacked up a working tool directory here:
>>
>> http://tools.wmflabs.org/hay/directory
>>
>> Try searching for stuff, clicking on the labels. To add your own tool,
>> scroll the page down for instructions.
>>
>> Current this only lists my tools, but i hope that this directory will
>> soon grow with everything the Wikimedia community has to offer.
>>
>> I'm interested in your opinion in this proposal and, if you like it,
>> add a toolinfo.json to your project!
>>
>> Kind regards,
>> -- Hay / [[User:Husky]]
>>
>> 1: https://en.wikipedia.org/wiki/Wikipedia:Tools
>> 2: https://www.npmjs.org/
>> 3: https://developer.chrome.com/apps/manifest
>>
>> ___
>> Labs-l mailing list
>> lab...@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/labs-l
>>
>
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Labs-l] A proposal for better tool discoverability

[Magnus Manske]

Thanks Husky,

I've added WDQ to try it. At what interval does it refresh?

It might be helpful if I could structure (group) tools on the wiki page.
Does the directory ignore not-well-formatted lines (e.g. headings)?

Cheers,
Magnus



On Wed, Aug 13, 2014 at 4:15 PM, Hay (Husky)  wrote:

> Hey everyone,
> for those who attended Wikimania, i hope you all had a wonderful time!
>
> One of the things that i've talked about at Mania with a few people is
> the current way of 'tool discovery', or rather, the lack thereof.
> We've seen really cool things done by the community, but i have a
> feeling many of the tools go by unnoticed, simply because nobody knows
> that they exist. Maybe they've been send around here or posted on IRC,
> but after the initial interest many of them are forgotten and lead a
> lonely life on our virtual servers.
>
> I think this is especially problematic for non-community members for
> whom the tools might be very useful. For example, as a Wikipedian in
> Residence and GLAM advocate i have sent numerous mails with links to
> the various GLAM tools made by Magnus, stats.grok.se, etcetera.
>
> But, you might say, we run all these knowledge sites called wikis
> right? And there are some pages there that list tools, don't they?
> You're right, the community has put some efforts in a few pages (1).
> However, IMHO, a wiki doesn't lend itself to easy tool discovery:
>
> * The format on how to describe a tool is unclear, thereby leading to
> both very long and very short descriptions.
> * It's not possible to easily search for tools in a certain category
> (e.g. 'GLAM' or 'editcount').
> * The categorization of tools leads to much clicking around,
> distracting and confusing users.
> * There's no way to sort results, for example by the number of people
> using the tool.
>
> The biggest problem of all is the disconnection between the actual
> code of the tool itself (hosted in a Git repo), and the description
> advertising it. People update their tool to do something different (or
> deprecate it), but they forget the documentation on the wiki, don't
> remove it when the tool doesn't work anymore, etcetera.
>
> I believe we can, and should do better.
>
> THE PROPOSAL
>
> My proposal is to use the same mechanisms that already exist for
> package managers such as NPM (2) or 'app stores', such as the Chrome
> store (3). Basically this involes adding a small JSON file to your
> project, including a few key properties such as 'title' and
> 'description'. These files get indexed, and an easy to use frontend to
> search to all the tools is provided to end users. You could imagine a
> 'toolinfo.json' file to look something like this:
>
> {
> "name" : "WikiDataQuery",
> "description" : "An API for Wikidata items and properties.",
> "url" : "http://wdq.wmflabs.org/";,
> "keywords" : "wikidata, api, query",
> "author" : "Magnus Manske"
> }
>
> If you have a web-hosted tool, simply stick it in the root of your
> tools directory so that it's reachable by the crawler. Whenever your
> tool data changes, just update the file and the directory will
> automatically update the directory site.
>
> The link to your toolinfo.json could be added to a Wiki page so that
> it's easy to remove your tool from the directory or change the URL to
> the JSON file.
>
> I'm a firm believer in putting code where your mouth is, so i've
> hacked up a working tool directory here:
>
> http://tools.wmflabs.org/hay/directory
>
> Try searching for stuff, clicking on the labels. To add your own tool,
> scroll the page down for instructions.
>
> Current this only lists my tools, but i hope that this directory will
> soon grow with everything the Wikimedia community has to offer.
>
> I'm interested in your opinion in this proposal and, if you like it,
> add a toolinfo.json to your project!
>
> Kind regards,
> -- Hay / [[User:Husky]]
>
> 1: https://en.wikipedia.org/wiki/Wikipedia:Tools
> 2: https://www.npmjs.org/
> 3: https://developer.chrome.com/apps/manifest
>
> ___
> Labs-l mailing list
> lab...@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/labs-l
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] html2wiki expired

[Magnus Manske]

on it


On Mon, Jan 13, 2014 at 8:23 AM, Federico Leva (Nemo) wrote:

> I notice https://toolserver.org/~diberri/cgi-bin/html2wiki/ expired: can
> someone port it to Labs (or even just temporarily resurrect in on
> Toolserver), please?
> I know there's also http://tools.wmflabs.org/magnustools/html2wiki.php ,
> but I usually try both and pick the best output; Magnus' tools is IIRC
> considered best for tables, I often got better results from the others.*
> Thanks,
> Nemo
>
> (*) Example with one huge page from a dozen of messy etherpads: <
> https://www.mediawiki.org/w/index.php?title=Language_
> portal/Daily/2012%E2%80%932013&action=history>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: https://wiki.toolserver.org/
> view/Mailing_list_etiquette




-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Need Tool Labs migration support?

[Magnus Manske]

On Thu, Oct 10, 2013 at 5:58 PM, Marc A. Pelletier  wrote:

> On 10/10/2013 12:32 PM, Silke Meyer wrote:
> > ok, I consulted Nosy, DaB. and Mette about this proposal and we will not
> > make the home directories accessible to others. They can contain all
> > sorts of private data, user names, e-mail addresses, ssh keys, who knows
> > what else - maybe even gpg keys, so handing the complete package over is
> > no option. I wouldn't want to "intrude" into the people's stuff. And
> > we'd get in trouble with the law here, as the German data protection act
> > is binding for us.
>
> I agree with this as well, for those very reasons.  This is the reason
> why Tool Labs insists that any "real" tool is run from multi-maintainer
> project equivalents, and with clear licensing.
>
> I would not allow user's own home directories to be turned over to a
> third party under any circumstances, this is why we want no code running
> there we might want to salvage in the future.  And, likewise, I'm pretty
> sure the foundation wouldn't /want/ to be given custody over data that
> may or may not be private from contributors who have not given explicit
> leave to do so.
>
> To be clear, I was not suggesting to just dump all home directories on
bittorrent ;-)

public_html and cgi-bin should contain most code, and for small tools,
could be checked quickly by hand for personal/secret data.

Also, of course I prefer people moving their own tools! Just in the
important-tool-will-die-with-toolserver scenario...

Magnus
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Need Tool Labs migration support?

[Magnus Manske]

It is annoying, but to be fair, it's "only" been this bad since yesterday
or so. Usually it's OK.


On Thu, Oct 10, 2013 at 11:50 AM, Christian Thiele  wrote:

> Hi,
>
> Am 09.10.2013, 14:55 Uhr, schrieb Magnus Manske <
> magnusman...@googlemail.com>:
>
>
>  There are some tools from e.g. apper that would probably move quite well.
>>
>
> I just started moving some simple tools a few days ago, but the
> availability seems very bad to me. Sometimes the tools work, sometimes not
> - at the moment nothing works for me, not only my tools but also e.g.
> http://tools.wmflabs.org/add-**information/<http://tools.wmflabs.org/add-information/>or
> http://tools.wmflabs.org/**catfood/ <http://tools.wmflabs.org/catfood/>
>
> It was told that the toolserver will die one year after "Tool Labs has all
> necessary features to replace the Toolserver.". One feature is availability
> - and for now it's impossible for me to move tools if they only work 50% of
> the day. So either the availability will be at nearly 100% or the shutdown
> will be delayed.
>
> Regards,
> Christian / APPER
>
>
> __**_
> Toolserver-l mailing list 
> (Toolserver-l@lists.wikimedia.**org
> )
> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l<https://lists.wikimedia.org/mailman/listinfo/toolserver-l>
> Posting guidelines for this list: https://wiki.toolserver.org/**
> view/Mailing_list_etiquette<https://wiki.toolserver.org/view/Mailing_list_etiquette>
>



-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Need Tool Labs migration support?

[Magnus Manske]

Interesting.

Most of this seems to be related to OSM/geo-something. Not sure how much
could actually be ported to Labs at the moment.

There are some tools from e.g. apper that would probably move quite well.

Also, there is geohack, which I had moved to Labs quite a while ago (
http://tools.wmflabs.org/geohack/ ). Not sure why it's not redirected/URLs
on Wikipedia haven't changed yet.


On Wed, Oct 9, 2013 at 1:26 PM, Marlen Caemmerer <
marlen.caemme...@wikimedia.de> wrote:

> Hello,
>
> reading this I feel forced to publish my short Top50 usage of yesterday
> URLs from Toolserver.
>
> Its just a quick hack on my console but I guess it is more clear output
> then probably a webalizer could offer.
> I just counted the lines in my access log from yesterday of how often a
> user dir was called.
> This is my command line: zcat access.log.1.gz|cut -d \/ -f 4,5|cut -d \?
> -f 1|sort |uniq -c|sort -rn
>
>
> 4410444 tiles/hikebike
> 3410590 ~cmarqu/hill
> 3030592 ~dschwen/wma
> 1568706 tiles/osm
>  651872 tiles/osm-labels-ru
>  452436  (should be toolserver.org)
>  242462 ~dapete/ime
>  191536 ~kolossos/openlayers
>  172239 ~para/geoip.fcgi HTTP
>  146808 ~kolossos/geoworld
>  117295 ~dschwen/iip
>  114200 tiles/bw-mapnik
>  112167 ~para/region.php
>  104377 tiles/osm-no-labels
>   99926 ~apper/pd
>   85028 ~osm/libs
>   63087 ~apper/sc
>   61585 ~dispenser/temp
>   59166 ~dapete/random
>   53725 ~dispenser/cgi-bin
>   50095 ~para/GeoCommons
>   48399 ~geohack/geohack.php
>   46208 ~lvova/cgi-bin
>   43747 images/wikimedia-toolserver-**button.png HTTP
>   30192 ~tparis/autoedits
>   26460 ~geohack/siteicon.png HTTP
>   24401 ~master/osmjson
>   23565 ~dispenser/ghct
>   22924 ~cmarqu/opentiles.com
>   22199 ~mzmcbride/redirector
>   22089 ~daniel/potd
>   21928 geohack/geohack.php
>   20984 ~para/geoip.fcgi
>   20635 ~chm/whois.php HTTP
>   16343 ~quentinv57/sulinfo
>   15717 ~krinkle/I18N
>   11955 ~daniel/WikiSense
>   11533 ~soxred93/images
>   11007 ~authoritycontrol/redirect
>9610 ~dapete/rss
>8677 favicon.ico HTTP/1.1" 200 1150 -
>8492 %7Egeohack/geohack.php
>7913 tsthumb/tsthumb
>7707 tiles/osm-labels-de
>7460 ~rhaworth/os
>7363 ~dispenser/resources
>6789 ~para/cgi-bin
>6561 tiles/osm-labels-en
>6543 ~dapete/wikinews-rss
>6205 ~dapete/ImageMapEdit
>
>
>
> __**_
> Toolserver-l mailing list 
> (Toolserver-l@lists.wikimedia.**org
> )
> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l
> Posting guidelines for this list: https://wiki.toolserver.org/**
> view/Mailing_list_etiquette
>



-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Need Tool Labs migration support?

[Magnus Manske]

On Tue, Oct 8, 2013 at 7:47 PM, DaB.  wrote:

>  We do not force people to license their
> stuff as open source (and when I read something like "we force them to
> move", then I’m glad we didn’t).
>
> Because then we could just make a copy of the tool in that nasty
open-source way of Wikipedia, instead of having it die with the toolserver?
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Need Tool Labs migration support?

[Magnus Manske]

Don't we have to state the license for all our code to get a password
renewal? I remember setting that at some point...

In any case, having a list of most accessed tools would be a start. At the
very least we could determine if the licensing is a major issue.


On Mon, Oct 7, 2013 at 11:34 PM, Marc A. Pelletier  wrote:

> On 10/07/2013 06:27 PM, Magnus Manske wrote:
> > * Forcibly port the most important, unmaintained tools to Labs
>
> Sadly, that's only possible when the code is unambiguously licensed as
> Open Source, which is often not the case (much of the code running on
> the Toolserver has no licensing information at all).
>
> -- Marc
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Need Tool Labs migration support?

[Magnus Manske]

Maybe this just needs a little more planning.
* Look at the server logs and find the most used tools on toolserver
* Contact those tool authors directly
* Make copies of the source of those tools publicly available (they should
all be open source, but some directories are not accessibly for other users
from shell IIRC)
* Keep a Foundation-readable snapshot of the toolserver home directories,
user-store, DB dumps etc.
* Forcibly port the most important, unmaintained tools to Labs
I have already started with one or two, though I'd appreciate it if I don't
have to do it all by myself ;-)


On Mon, Oct 7, 2013 at 8:31 PM, Maarten Dammers  wrote:

> Hi Silke,
>
> I'm afraid you're effort won't have the intended outcome. What I expect
> will happen:
> * Some tool authors will move in the next couple of months
> * Some tool authors will move at the last moment
> * A lot of tools willl just be abandoned.
> * You'll get a lot of upset people when you shut down the Toolserver
>
> Don't say I didn't warn you,
>
> Maarten
>
> Op 7-10-2013 14:03, Silke Meyer schreef:
>
>  Hey all!
>>
>> We had set a deadline (Sept 30th) to ask WMDE for support to migrate
>> your tools to Tool Labs. As very few of you made use of this until now,
>> Johannes still has time to help you. So we are still accepting support
>> requests. Please think about whether you'll have the time and the
>> know-how to migrate on your own. Your tools shall have left the
>> toolserver until June 30th, 2014. This deadline is serious.
>>
>> If you would like Johannes to give you a hand, please tell us until
>> November 15th 2013.
>> * Write to me or to directly to johannes.kr...@wikimedia.de.
>> * Give some details about your tool: Did you try to migrate already but
>> something fails to work? What works? What doesn't? What exactly do you
>> want Johannes to do?
>>
>> Best, Silke
>>
>>
>
>
> __**_
> Toolserver-l mailing list 
> (Toolserver-l@lists.wikimedia.**org
> )
> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l
> Posting guidelines for this list: https://wiki.toolserver.org/**
> view/Mailing_list_etiquette
>



-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Maintainer(s) wanted: Contribs, new pages and others

[Magnus Manske]

It has begun:
https://meta.wikimedia.org/wiki/User_talk:Erwin#Labs_port

Anyone else interested, please help me porting/maintaining; grunt work is
done ;-)



On Fri, Sep 27, 2013 at 11:22 AM, Silke Meyer wrote:

> Erwin85 is looking for people to maintain his very popular tools [1] (as
> he wrote on [2]). They are currently running on the toolserver. If you are
> looking for a project to help with, please have a look at the tools.
>
> If you are interested in helping to maintain one or a few of them, here
> are your next steps:
> * Contact Erwin85. ;)
> * Ask in Jira [3] to have the tool you want to work on transformed into a
> multi-maintainer project and have yourself added to it.
> * Migrate them to Tool Labs. (WMDE can help here, if you want. Contact me!)
> * Take over responsibility for long term maintenance of one or several of
> the tools.
>
> Cheers, Silke
>
> [1] https://toolserver.org/~erwin85/
> [2] https://meta.wikimedia.org/wiki/User_talk:Erwin#Tool_Labs
> [3] https://jira.toolserver.org
>
> --
> Silke Meyer
> Internes IT-Management und Projektmanagement Toolserver
>
> Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
> Tel. (030) 219 158 260
>
> http://wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter
> der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für
> Körperschaften I Berlin, Steuernummer 27/681/51985.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>



-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] TS and/vs. Labs

[Magnus Manske]

On Tue, Aug 13, 2013 at 6:51 AM, Maarten Dammers wrote:

> Hi DrTrigon,
>
> Op 10-8-2013 11:36, Dr. Trigon schreef:
>
>  -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> Hello DaB!
>>
>> Could you please give me a short update - the end of 2013 is
>> approaching, what is the current state for TS resp. Labs?
>>
> If you don't care about losing your (nfs) file system every once in a
> while and having very slow databases, you can probably start moving stuff
> to toollabs.
> I'm waiting for these bugs to be fixed. It's been more than 3 months now
> and no clear estimate when it will be fixed.
>
>
I don't think that's a fair representation. Despite some teething problems,
Labs has been at least (probably more) reliable than the toolserver in the
time I've been there. I have ported 25 tools to Labs now, including some
heavy-duty ones, and they all seem to work fine.

Yes, there is the occasional bug (imagelinks table on cawiki is not working
at the moment; roots are en route from Wikimania;-) but I don't see the
traditional toolserver slowness, and Labs doesn't occasionally vanish
behind 504 errors for hours at a time either.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] TS and/vs. Labs

[Magnus Manske]

If you're asking "is it safe to move tools to Labs", my experience so far
say "yes". I can't speak for all unique toolserver features, though.

Cheers,
Magnus


On Sat, Aug 10, 2013 at 10:36 AM, Dr. Trigon  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hello DaB!
>
> Could you please give me a short update - the end of 2013 is
> approaching, what is the current state for TS resp. Labs?
>
> Thanks and Greetings
> DrTrigon
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.13 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iEYEARECAAYFAlIGCboACgkQAXWvBxzBrDAKPwCgjsEcGfRtbKaw/eDwTxNhUFfo
> XgwAoLzV6J+TCdJZ3lvwvkqsMWihQmr+
> =1dN+
> -END PGP SIGNATURE-
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette




-- 
undefined
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Status update on Tool Labs

[Magnus Manske]

Slightly OT:
Sorry if I missed that in the discussion, but is there a for-dummies howto
for creating a labs git for the ported tool? I know they use gerrit, but
that's the extend of my knowledge.


On Sun, Jun 2, 2013 at 9:22 PM, Merlijn van Deen wrote:

> On 2 June 2013 21:12, Tim Landscheidt  wrote:
> > Maciej Jaros  wrote:
> >
> >> Is there any how-to on moving SVN repo. along with it's
> >> history to TL? Is it possible/planned at all?
> >
> > As DaB. explained, the roots can provide you with a dump of
> > your SVN repository.  You can then import ("load") this into
> > a repository in your home directory and access it via ssh
>
> Note: you can also use svnsync to do this from your own computer - see
> https://github.com/pywikibot/svn2git/blob/master/scripts-pwb/sync for
> an example.
>
> Merlijn
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Status update on Tool Labs

[Magnus Manske]

Are they up somewhere else? Otherwise, redirects won't be useful.

I tried to have a look at interiot's tools, but both web and shell access
are blocked. There might be volunteers to port abandoned tools to labs (as
I did for [1]), but only if they are still sought-after, and if the source
code is accessible and properly licensed.

[1] http://tools.wmflabs.org/joanjoc/


On Fri, May 31, 2013 at 9:19 AM, Federico Leva (Nemo) wrote:

> Would .htaccess work also for expired accounts then, and could
> operators/roots do something about them? We still have tons of users
> entering, say, interiot and soxred's tools and getting slightly unhelpful
> results.
>
>
> Nemo
>
> __**_
> Toolserver-l mailing list 
> (Toolserver-l@lists.wikimedia.**org
> )
> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l
> Posting guidelines for this list: https://wiki.toolserver.org/**
> view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Status update on Tool Labs

[Magnus Manske]

Database HOWTO:
https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolserver#Database_access



On Mon, May 27, 2013 at 4:27 PM, Silke Meyer wrote:

> Hello to all tool maintainers!
>
> It was good to see several of you in person this weekend! Faces ftw! :)
>
> Here is a status update with what is now working in Tool Labs:
>
> First of all (looking at DaB.s survey and your comments):
> You _don't_ have to care about virtual instances, puppet or any of
> this. This is underneath/behind Tool Labs but you can just use the
> login server without having to know about it.
>
> What you get:
> * login server made to resemble the toolserver
> * database replicas from s1-s6, s7 is following in June. Access to all
> data you can access as a registered, unprivileged user.
> * possibility to create your own user databases
> * shared project storage, home directories (nfs)
> * possibility to add maintainers to your project via the web interface
> on http://tools.wmflabs.org/
> * possibility to restore your files to the state they were in 1, 2, 3
> hours ago, 1, 2, 3 days ago and on the last 2 Sundays.
> * infrastructure to run web services
> * job system: OpenGridengine
> * two different environments to develop tools in: "Tools" project
> which is a stable environment, "Bots" project which can be used for
> experiments with the environment, root access is possible in the
> latter.
> * bug tracking in WMF's Bugzilla (product: Wikimedia Labs, components:
> bots and tools for the two projects)
> * more stuff that is mentioned in the road map (link below).
>
> What we are still on:
> * documentation (let's crowdsource this!)
> * make Merlissimo's MerlLinkBot hunt down deprecated weblinks and
> point them to your tools' new URL.
>
> Looking for things you can help with?
> * Write down you experiences like some already did: Magnus Manske:
>
> https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolserver
> or Russell Blau:
>
> https://wikitech.wikimedia.org/wiki/User:Russell_Blau/Using_pywikibot_on_Labs
> * Help to improve the user interface of tools.wmflabs.org! Contact
> Coren/Marc-Andre Pelletier.
>
> Links:
> Tool Labs help:
> https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Help
> FAQ (ongoing collection of what you ask):
> http://www.mediawiki.org/wiki/Wikimedia_Labs/Migration_of_Toolserver_tools
> Road map: http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_en (English)
> and http://www.mediawiki.org/wiki/Tool_Labs/Roadmap_de (German)
>
> What's next?
> Please come and try out the new database replication!
> If your tool is up and running in Tool Labs, _please consider
> documenting_ what you missed in the docs. By doing so you will help
> others a lot!
> Don't hesitate to ask! The best places to ask questions:
> * IRC (#wikimedia-labs), where Coren and petan (as Tool Labs admins)
> and many other experienced people hang around
> * the list labs-l (https://lists.wikimedia.org/mailman/listinfo/labs-l)
>
> Do your tools need more adaptations than you think you can do?
> Wouldn't mind a helping hand? Ask me and I'll try to find you a
> helper!
>
> Best, Silke
>
> --
> Silke Meyer
> Internes IT-Management und Projektmanagement Toolserver
>
> Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
> Tel. (030) 219 158 260
>
> http://wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
> Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Status update on Tool Labs

[Magnus Manske]

.htaccess is your friend :-)


On Tue, May 28, 2013 at 7:59 AM, Federico Leva (Nemo) wrote:

> A bot to update URLs is a nice thing (except that some stupid wikis will
> surely block it), but a true solution would be setting up a system where
> redirects from toolserver to new URLs can be set up without too much pain.
>
> Nemo
>
>
> __**_
> Toolserver-l mailing list 
> (Toolserver-l@lists.wikimedia.**org
> )
> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l
> Posting guidelines for this list: https://wiki.toolserver.org/**
> view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Migrating to labs from the toolserver

[Magnus Manske]

.htaccess redirects?


On Sat, May 4, 2013 at 2:10 PM, rupert THURNER wrote:

> What happens to the zillions of published toolserver links when a tool
> switches to labs?
>
> Rupert
> Am 04.05.2013 14:18 schrieb "Magnus Manske" :
>
>>  As I boldly ssh in where few toolserver users have ssh'ed in yet, I try
>> to keep a log of what I'm doing, what issues arise, and where I had to stop
>> and scratch my head. Behold:
>>
>>
>> https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolserver
>>
>> Feel free to improve the page as you see fit.
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Migrating to labs from the toolserver

[Magnus Manske]

As I boldly ssh in where few toolserver users have ssh'ed in yet, I try to
keep a log of what I'm doing, what issues arise, and where I had to stop
and scratch my head. Behold:

https://wikitech.wikimedia.org/wiki/User:Magnus_Manske/Migrating_from_toolserver

Feel free to improve the page as you see fit.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikidata tables

[Magnus Manske]

FWIW, I have implemented a query-able stand-alone web server that keeps all
of the wikidata property-item-links in memory. This uses the wikidata dumps
which appear to be rather frequent. I'll try do deploy a test version on
wikilabs (once I figure out how all that works); it seems to be more
favourable to such services than the toolserver.


On Fri, Apr 19, 2013 at 9:29 AM, Platonides  wrote:

> On 19/04/13 01:19, DaB. wrote:
> > as you may know there is a rev_text_id-field in the revision-table. This
> field
> > points to the text-table where the actual text is – or should be.
> Because the
> > WMF doesn’t store the text here, but only a pointer
> ("DB://cluster25/11458305"
> > for example). If you query different wikis you will see that most of
> them point
> > to the same cluster or one with a number short by. That says me (and I
> was
> > also told so before) that all text of all wmf-projects are stored
> together.
> > The task would now to separate wikidata from the rest – but the
> storage-area
> > has no clue from where a text is which makes the separating very hard.
> And
> > there is another problem: Deleted texts are also in this area, so even
> more
> > filtering would be needed.
> > I very doubt that this situation will change at the TS and I also doubt
> that
> > it will be different for WikiLabs. So I guess your best bet is the API
> here.
> >
> > Sincerely,
> > DaB.
>
> I think the only hope would be if wikidata was stored under its own
> cluster (for easier differenciation) and at least one server of that
> group (the master?) only had that (so toolserver could get its binlogs).
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikidata tables

[Magnus Manske]

Huh. That could be ... problematic in the future.

Thanks,
Magnus


On Thu, Apr 18, 2013 at 10:21 AM, Lydia Pintscher <
lydia.pintsc...@wikimedia.de> wrote:

> On Thu, Apr 18, 2013 at 9:52 AM, Magnus Manske
>  wrote:
> > Just wondering what the status of exposing all wikidata tables on the
> > toolserver is.
> >
> > Currently, there are a few wb_* tables with item labels, descriptions,
> > aliases, and language links.
> >
> > But the tables (whatever they are called) containing item-to-item
> > connections appear to be missing. Maybe because they were added later?
>
> As far as I know they're only saved in JSON where usually the article
> text is stored and not in separate tables.
>
>
> Cheers
> Lydia
>
> --
> Lydia Pintscher - http://about.me/lydia.pintscher
> Community Communications for Wikidata
>
> Wikimedia Deutschland e.V.
> Obentrautstr. 72
> 10963 Berlin
> www.wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
>
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das
> Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Wikidata tables

[Magnus Manske]

Just wondering what the status of exposing all wikidata tables on the
toolserver is.

Currently, there are a few wb_* tables with item labels, descriptions,
aliases, and language links.

But the tables (whatever they are called) containing item-to-item
connections appear to be missing. Maybe because they were added later?

Magnus
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Toolserver down?

[Magnus Manske]

Can't get http connection, and ssh says:

ssh: Could not resolve hostname login.toolserver.org: nodename nor servname
provided, or not known
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Can't log into toolserver

[Magnus Manske]

Keeps asking for password, but doesn't accept it. login.toolserver.org,
also ortelius, probably more.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Tool request

[Magnus Manske]

Why would you want to upload downsized images if you can just have
MediaWiki generate downsized thumbnails?


On Fri, Mar 22, 2013 at 7:05 PM, John  wrote:

> This is slightly outside the scope of this mailing list, But could
> someone write a web tool that someone can pass a filename, lang,
> project and the script then gets the associated image, lets the user
> modify the size (and has a few pre-configured resolutions including
> 0.1MP ) then download it so it can be re-uploaded (or uploaded via
> upload by url if the wiki has it enabled.)
>
> I know this would help a lot of users.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Hello, and the short term plans

[Magnus Manske]

It appears they are not starting "from scratch", but to set up an
environment as familiar as possible for the tools, within the inevitable
changes that come with such a migration. "The fewer changes needed to
existing tools the better" from the Design page is a hint in that direction
;-)


On Mon, Feb 25, 2013 at 10:30 PM, Maarten Dammers wrote:

> Hi Marc,
>
> Why start from scratch again? Why not just migrate the current Toolserver
> setup to the WMF cloud infrastructure?
>
> Maarten
>
> Op 25-2-2013 18:52, Marc A. Pelletier schreef:
>
>> Good news, everyone!
>>
>> As many of you know, I officially started my duties today as the WMF
>> Operations Engineer attached to the Tool Labs.  I intend to make a point of
>> informing all of you of recent news, what I'm working on, and where I'm
>> headed at regular intervals (probably weekly).
>>
>> First, a bit of news:  I have had confirmation this weekend that the DB
>> replication made available to Tool Labs users will, in fact, allow the
>> creation of databases alongside the project ones. This means that one of
>> the use cases that seemed the most troublesome in the transition (joins
>> between the WMF databases and tool-specific ones) will be fully supported.
>>
>> We are making good strides in documenting the *impressive* inventory of
>> tools that run on toolserver and their requirements (thanks, Silke!).  The
>> list-in-progress can be found at [1].  If you see missing or incorrect
>> information, please feel free to adjust it -- the more precisely we know
>> the requirements, the faster we can see about meeting them.
>>
>> I've started documenting my preleminary design for the shiny new Tool
>> Labs infrastructure at [2].  This is a living document, and will see a
>> great deal of revision before it's over (and will serve as the seed for the
>> documentation).  I will shortly create a new Labs project where that
>> architecture is deployed in preproduction so we can shake out the kinks.
>>  The existing projects, "bots" and "webtools" will be left active for the
>> forseeable future until (a) the new architecture has proven itself and (b)
>> every user has sucessfuly moved their tools to it.
>>
>> I'm planning on having the new project be fully operational for new tools
>> by the time the Amsterdam Hackathon takes place at the end of May at the
>> very least.
>>
>> For the next week, I'll be mostly in information-gathering mode, as well
>> as refining the design and requirements of the Tool Labs. Feel free to poke
>> me for information (or /with/ information) by email or on IRC (where I am
>> user 'Coren' and idle on #wikimedia-labs and #wikimedia-toolserve at the
>> very least)
>>
>> -- Marc A. Pelletier
>>
>> [1] 
>> http://www.mediawiki.org/wiki/**Toolserver/List_of_Tools
>> [2] 
>> http://www.mediawiki.org/wiki/**Wikimedia_Labs/Tool_Labs/**Design
>>
>>
>>
>> __**_
>> Toolserver-l mailing list 
>> (Toolserver-l@lists.wikimedia.**org
>> )
>> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l
>> Posting guidelines for this list: https://wiki.toolserver.org/**
>> view/Mailing_list_etiquette
>>
>
>
>
> __**_
> Toolserver-l mailing list 
> (Toolserver-l@lists.wikimedia.**org
> )
> https://lists.wikimedia.org/**mailman/listinfo/toolserver-l
> Posting guidelines for this list: https://wiki.toolserver.org/**
> view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] FYI: TS wikivoyage is missing a few pages

[Magnus Manske]

https://jira.toolserver.org/browse/TS-1598
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Hello! New TS coordinator @ WMDE

[Magnus Manske]

Hi Silke,

welcome to the crucible! :-)

Magnus


On Fri, Feb 1, 2013 at 2:08 PM, Silke Meyer wrote:

> Dear Toolserver people!
>
> Hi! I am the new person managing all toolserver issues at Wikimedia
> Germany. This means that I am the contact person at WMDE for questions
> concerning the current toolserver and I coordinate a small team of WMDE
> staff members for the upcoming migration process to WMF Labs.
>
> Wikimedia Germany is going to support the toolserver as long as there is
> no alternative ready. I would like to stress that I'll do my best to
> help you keep it running until then. Please don't hesitate to contact me
> when any problem occurs. I am in touch with DaB and Marlen and I will
> help with all I can do at WMDE to reduce failures/the time it takes to
> fix them.
>
> As you know, Wikimedia Foundation is currently working on a cloud-based
> infrastructure for tools that will - at some point in the nearer future
> - have functionalities equivalent to the toolserver. Many parts of it
> are up and running, but right now, it is not clear when Labs will have
> all the functionalities. Currently, WMDE and WMF are working together to
> set up a concrete road map within the next months (until May). Once this
> road map or further documentation exist, you'll know it, of course.
>
> When it comes to migrating your stuff, WMF and WMDE staff will support
> you as best we can - you saw Sumana's e-mail on this topic recently. I
> am looking forward to working with you all!
>
> You'll hear from me again soon and please don't hesitate to contact me
> if you have any questions!
>
> Best,
> Silke
>
>
> --
> Silke Meyer
> Systemadministratorin und Projektassistenz Wikidata
>
> Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin
> Tel. (030) 219 158 260
>
> http://wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt
> für Körperschaften I Berlin, Steuernummer 27/681/51985.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Result of the general member meeting of WMDE

[Magnus Manske]

Doubleplusgood for DaB staying in the fight for another round!!!


On Mon, Dec 3, 2012 at 12:49 AM, DaB.  wrote:

> Hello all,
> Am Sonntag 25 November 2012, 23:44:05 schrieb DaB.:
> > I have not decided yet if I will remain as root under this circumstances
> > for  2013 – I will tell you my decision until next Sunday.
>
> I promised you a decision until Sunday and while it is not Sunday anymore
> where I live, for many of you still is.
> First of all I would like to thank all people who have sent me emails:
> Thanks.
>
> During the last week I thought about my decision several times and to be
> honest I switched from staying to leave and back several times.
> The toolserver consumes a big part of my free time and so the idea of
> leaving
> was alluring. It would also de-stress me, because I would not longer need
> to
> argue with people from WMDE and WMF. Maybe even the personal relationship
> with
> some people of WMDE would normalize again.
> On the other hand the toolserver is a important project into which I
> invested
> much time already. Many people would dislike me if I go down without a
> fight
> too. The toolserver is also important for WMDE and its relationship to WMF
> –
> even if most people do not see that at the moment.
> Another path of thought was optimism versus pessimism. The pessimist in me
> says, that it is very sure that WMDE will find a cheap way out in 6 months
> when
> it becomes clear that WikiLabs can not replace the toolserver – but I have
> no
> proof for that. The optimist in me says that that is most likely but it
> COULD
> be that WMDE is fair and the 6 months of waiting are worth that.
> The third point is the question of motivation. I really like to help
> people –
> that's the reason I started editing and sysoping in Wikipedia 9 years ago.
> The
> idea that the toolserver helps people to do their work is the basic
> motivation
> for me to administrate the toolserver. But when you know that all you do is
> for the birds because some other people decided that the toolserver is not
> welcome anymore that's not very motivational. Than again helping people for
> 6-12 months is better than nothing.
> The forth, least significant and last point is the question of pride.
> Leaving
> would be a form of loosing; WMDE and WMF would win. I don't like loosing
> and I
> prefer to fight till the end for normal.
>
> To summarize (people who scrolls down to find the result should stop
> here): I
> will stay for another 12 months and see where it goes. The toolserver and
> you
> all are worth my time, the toolserver is still helpful, and there is always
> the possibility that WMDE will do a fair decision in summer. The way will
> be
> rocky and I'm sure that we will see some drawbacks.
>
> Good night everybody.
>
> Sincerely,
> DaB.
>
>
>
>
> --
> Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Anyone else having TS trouble?

[Magnus Manske]

Works for me now.


On Sun, Nov 25, 2012 at 6:19 PM, Krinkle  wrote:

> On Nov 25, 2012, at 5:06 PM, Platonides  wrote:
>
> On 25/11/12 16:42, Magnus Manske wrote:
>
> Started a few minutes ago. getting alternating errors on reloading a
> page; 500 (on a static HTML page, not CGI, mind you!), "This webpage is
> not available: The connection to toolserver.org <http://toolserver.org>
> was interrupted.", 404.
>
>
> The same thing happpened here.
> The servers didn't allow logins, and accessing the web pages return
> 404s. Apparently ldap was down.
> It recovered now. I don't know if it there was a human involved, if it
> solved itself or if it was turnera taking over damiana automatically.
>
>
> Seems fine here:
>
> https://toolserver.org/~krinkle/mwSnapshots/#!/mediawiki-core/master
>
> Maybe HTTP or Geo related? Is anyone still having this problem?
>
> -- Krinkle
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Anyone else having TS trouble?

[Magnus Manske]

Started a few minutes ago. getting alternating errors on reloading a page;
500 (on a static HTML page, not CGI, mind you!), "This webpage is not
available: The connection to toolserver.org was interrupted.", 404.
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] 502

[Magnus Manske]

Thanks all. Must be my line then.


On Mon, Nov 12, 2012 at 3:38 PM, Mardetanha wrote:

> no problem from Iran and through vpn connection from UK
>
> Mardetanha
>
>
> On Mon, Nov 12, 2012 at 7:03 PM, Simon Walker  > wrote:
>
>> No problems from Edinburgh (Scotland)
>>
>> Simon
>> On Nov 12, 2012 3:26 PM, "Magnus Manske" 
>> wrote:
>>
>>> I suddenly see a lot of 502 error on the toolserver from the UK. Anyone
>>> else? (can't find a IRC client for OSX 10.6 quickly...)
>>> ___
>>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>>> Posting guidelines for this list:
>>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>>
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] 502

[Magnus Manske]

I suddenly see a lot of 502 error on the toolserver from the UK. Anyone
else? (can't find a IRC client for OSX 10.6 quickly...)
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Welcome: wikidata

[Magnus Manske]

Fantastic! Thank you!

On Tue, Oct 30, 2012 at 3:00 PM, DaB.  wrote:
> Hello all,
>
> wikidata has made it on the toolserver today. The database is called
> "wikidatawiki_p" (don't ask me…) and its is home on s3. Because I guessed that
> many joins to it will happen in future, I created also copies on all clusters
> with the exception of s6 and s7 (will happen later); so you can join it in the
> same way as commons.
>
> Sincerely,
> DaB.
>
> --
> Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Web services are down

[Magnus Manske]

Me too. Occasionally, it's "502 bad gateway" though. Just in the
middle of testing stuff and thought it was my broken code.
Frustration.



On Sun, Sep 2, 2012 at 4:33 PM, emijrp  wrote:
> I'm getting Internal Server Error (500) and pages dont load.
>
>
> 2012/8/29 Platonides 
>>
>> On 29/08/12 09:22, Marlen Caemmerer wrote:
>> > Hello,
>> >
>> > wolfsbane ran out of space which means one of the web servers was not
>> > able to deliver any more web pages which has the effect of "sometimes
>> > the page works - sometimes not".
>> > Should be fixed now - if you still see errors please let us know.
>> >
>> > Cheers
>> > nosy
>>
>> Thanks nosy.
>>
>> Shouldn't the balancer have automatically considered wolfsbane down and
>> direct all requests to ortelius?
>>
>> Cheers
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
>
>
>
> --
> Emilio J. Rodríguez-Posada. E-mail: emijrp AT gmail DOT com
> Pre-doctoral student at the University of Cádiz (Spain)
> Projects: AVBOT | StatMediaWiki | WikiEvidens | WikiPapers | WikiTeam
> Personal website: https://sites.google.com/site/emijrp/
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] What's up with enwp10?

[Magnus Manske]

Thanks!

On Wed, Aug 29, 2012 at 4:02 PM, DaB.  wrote:
> Hello,
> At Wednesday 29 August 2012 17:01:58 DaB. wrote:
>> Might read through the "[Toolserver-l] Web services are down" thread.
>
> no, read the we-disabled-our-own-tool-thread at [1] ;-).
>
> Sincerely,
> DaB.
>
> [1]
> https://en.wikipedia.org/wiki/Wikipedia_talk:Version_1.0_Editorial_Team/Index#WP_1.0_bot_temporarily_down_2012-8-7
>
> --
> Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] What's up with enwp10?

[Magnus Manske]

Hi all,

the "Wikipedia 1.0" tool gives a "403 forbidden" error:

http://toolserver.org/~enwp10/bin/list2.fcgi?run=yes&projecta=Computational_Biology&namespace=&pagename=&quality=&importance=&score=&limit=1500&offset=1&sorta=Importance&sortb=Quality+%28reverse%29

Does anyone know what's going on, or how to fix this?

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wpipe

[Magnus Manske]

You can access the asset metadata in the "u_magnus_wpipe_p" database,
table "asset", if you wish. The files are stored in
/mnt/user-store/wpipe/

Cheers,
Magnus


On Sun, Jul 22, 2012 at 10:51 AM, Dr. Trigon  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> To me as someone always struggling with sql queries (since I am not an
> expert at all) this sound somehow promising. May be there will be a
> comprehensive set of queries going to pipes one day. Then I would simply
> have to pick up the pipe and connect it further. (I like this flavour
> of UNIX philosophy ;)
>
> Greetings
> DrTrigon
>
> On 19.07.2012 22:34, Magnus Manske wrote:
>> On Thu, Jul 19, 2012 at 8:06 PM, Platonides 
>> wrote:
>>> I'm not convinced about its utility. What tools would need
>>> combining? If I just need the results of a SQL query, it may be
>>> easier for me than using this system. Maybe a better interface
>>> would help.
>>>
>>> The use case I see more interesting is for taking a tool which
>>> outputs a list of pages and provide for input of another tool.
>>> Some page/user to work with seems to be the most common input.
>>> Maybe we should just standarize the input parameters and let some
>>> tools chain to another, simply using a special format parameter.
>>>
>>> For instance I usually use names like: art, lang, project such
>>> as: $_REQUEST += array('art'=>'', 'lang'=>'en',
>>> 'project'=>'wikipedia' );
>>
>> I believe we mean the same thing; maybe I didn't describe the
>> asset thing very well.
>>
>> It's not for a "single page run" of some tool; one reason I chose
>> my CatScan rewrite as demo ist that it can run for a long time
>> (two-digit number of minutes), and generate a vast list of results
>> (tens of thousands of pages), depending on the query. The idea is
>> that (a) you're not "blocking" while waiting for that to finish,
>> before you can do something else; (b) you can access the results of
>> the run again, maybe if the subsequent tool fails, or you want to
>> try a different filter or subset, or a different subsequent tool
>> altogether; (c) you can define new data sources, maybe a tool where
>> you just paste in page titles, or another tool that gets the newest
>> 1.000 articles, or 1.000 random articles, or the last 1.000
>> articles you edited, or /insert crazy idea here/, and all
>> subsequent tools will just run with it.
>>
>> And you can chain tools together via a single number; no file path
>> that the other guy doesn't have access to, no sql query that runs
>> for a few minutes every time (that is, /if/ your tool can be
>> reduced to that...), no massive paste orgy, no loss of meta-data
>> between tools.
>>
>> I also envision longer chains: Give me all articles that are in
>> both these two category trees; remove the ones that have images
>> (except template symbol icons, if possible); remove the ones that
>> have language links; remove the ones that had an edit less than a
>> month old; render that as wikitext. There's a subject-specific
>> "needs work" finder from simple components. UNIX philosophy at its
>> finest :-)
>>
>>
>>> Magnus Manske wrote:
>>>> Right now, a tool is started by this page via "nohup &"; that
>>>> could change to the job submission system, if that's possible
>>>> from the web servers, but right now it seems overly complicated
>>>> (runtime estimation? memory estimation? sql server access?
>>>> whatnot) The web page then returns the reserved output asset
>>>> ID, while the actual tool is running; another tool could thus
>>>> be "watching" asynchronously, by pulling the status every few
>>>> seconds.
>>>
>>> Yes, it can be called. I use it in a script for scheduling a
>>> cleanup of the created temporary files.
>>>
>>> The relevant code:
>>>> $dt = new DateTime( "now", new DateTimeZone( "UTC" ) ); $tmpdir
>>>> = dirname( __FILE__ ) . "/tmp"; @mkdir( $tmpdir, 0711 ); $shell
>>>> = "mktemp -d --tmpdir=" . escapeshellarg($tmpdir) . "
>>>> catdown.";
>>>>
>>>> $tmpdir2 = trim( `$shell` ); // Program the folder destruction
>>>> // Note that qsub is 'slow' to return, so we perform it in the
>

Re: [Toolserver-l] Wpipe

[Magnus Manske]

On Thu, Jul 19, 2012 at 8:06 PM, Platonides  wrote:
> I'm not convinced about its utility. What tools would need combining?
> If I just need the results of a SQL query, it may be easier for me than
> using this system. Maybe a better interface would help.
>
> The use case I see more interesting is for taking a tool which outputs a
> list of pages and provide for input of another tool. Some page/user to
> work with seems to be the most common input.
> Maybe we should just standarize the input parameters and let some tools
> chain to another, simply using a special format parameter.
>
> For instance I usually use names like: art, lang, project such as:
>  $_REQUEST += array('art'=>'', 'lang'=>'en', 'project'=>'wikipedia' );

I believe we mean the same thing; maybe I didn't describe the asset
thing very well.

It's not for a "single page run" of some tool; one reason I chose my
CatScan rewrite as demo ist that it can run for a long time (two-digit
number of minutes), and generate a vast list of results (tens of
thousands of pages), depending on the query. The idea is that (a)
you're not "blocking" while waiting for that to finish, before you can
do something else; (b) you can access the results of the run again,
maybe if the subsequent tool fails, or you want to try a different
filter or subset, or a different subsequent tool altogether; (c) you
can define new data sources, maybe a tool where you just paste in page
titles, or another tool that gets the newest 1.000 articles, or 1.000
random articles, or the last 1.000 articles you edited, or /insert
crazy idea here/, and all subsequent tools will just run with it.

And you can chain tools together via a single number; no file path
that the other guy doesn't have access to, no sql query that runs for
a few minutes every time (that is, /if/ your tool can be reduced to
that...), no massive paste orgy, no loss of meta-data between tools.

I also envision longer chains: Give me all articles that are in both
these two category trees; remove the ones that have images (except
template symbol icons, if possible); remove the ones that have
language links; remove the ones that had an edit less than a month
old; render that as wikitext. There's a subject-specific "needs work"
finder from simple components. UNIX philosophy at its finest :-)


> Magnus Manske wrote:
>> Right now, a tool is started by this page via "nohup &"; that could
>> change to the job submission system, if that's possible from the web
>> servers, but right now it seems overly complicated (runtime
>> estimation? memory estimation? sql server access? whatnot)
>> The web page then returns the reserved output asset ID, while the
>> actual tool is running; another tool could thus be "watching"
>> asynchronously, by pulling the status every few seconds.
>
> Yes, it can be called. I use it in a script for scheduling a cleanup of
> the created temporary files.
>
> The relevant code:
>> $dt = new DateTime( "now", new DateTimeZone( "UTC" ) );
>> $tmpdir = dirname( __FILE__ ) . "/tmp";
>> @mkdir( $tmpdir, 0711 );
>> $shell = "mktemp -d --tmpdir=" . escapeshellarg($tmpdir) . " 
>> catdown.";
>>
>> $tmpdir2 = trim( `$shell` );
>> // Program the folder destruction
>> // Note that qsub is 'slow' to return, so we perform it in the background
>> $dt->add( new DateInterval( "PT1H" ) );
>> exec( "SGE_ROOT=/sge/GE qsub -a " . $dt->format("YmdHi.s") . " -wd " . 
>> escapeshellarg( $tmpdir ) . " -j y -b y /bin/rm -r " . escapeshellarg( 
>> $tmpdir2 ) . " 2>&1 &" );


Thanks, that looks interesting. I'll play with it, thou I still face
the problem of estimating resource requirements for a tool by a
generic wrapper. /Shudder/

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wpipe

[Magnus Manske]

On Thu, Jul 19, 2012 at 4:44 PM, Petr Onderka  wrote:
>> * Currently, data creation is limited to toolserver IP adresses (yes,
>> I know, it can be gamed. Like the rest of the wiki world.)
>
> This doesn't seem to be working.
> I was able to create asset 12 directly from my home computer.

I'll look into that. Not a big issue, I just don't want to make it too
easy for the script kiddies to store warez here...

> Also, action=fail doesn't work: "Error : Unknown action fail"
> It seems it's not listed in the if in index.php starting at line 23.

Thanks, should be fixed now.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Wpipe

[Magnus Manske]

Occasionally, "toolserver people" (both programmers and users) talk
about joining up tools. Wouldn't it be great if we could use one or
several toolserver tools, and "mash-up" their output to create
something new and useful? And wouldn't it be even better if the users
could do this directly, across tools, without programmers
hard-connecting tools?

Some tools already support machine-readable output, and some tools
already use others to perform a specific function. But these are
hardcoded, out formats are often crude (tabbed text, not that there's
anything wrong with that in principle), runtimes add up, and so on.

So, I went ahead and, as a first step towards a pipeline setup called
wpipe ("w" for "wiki", as you no doubt have guessed), implemented an
asset tracker. Here, an asset is a dataset, ideally a JSON file (I
also created a simple structure to hold lists of wiki pages, along
with arbitrary metadata). Each asset is tracked in a database,
accessible via a unique numeric identifier, and its data is stored in
a file. Assets can be created and queried via toolserver command line,
as well as via a web interface.

The usual steps in asset creation involve:
1. reserve (gets a new, unique ID)
2. start (set a flag that the asset data creation has begun)
3. done (store data, either by creating a file directly, or passing
the data to be stored)
4. fail (if there was an error during data creation)

Some points:
* All assets and associated data are public.
* Creation and last-access time (write or read the actual data) are
tracked, so unused assets can be removed to conserve storage.
* Currently, data creation is limited to toolserver IP adresses (yes,
I know, it can be gamed. Like the rest of the wiki world.)
* The suggested JSON format should be flexible enough for most tools
dealing with lists of wiki pages, but any text-based format will work
for specialist uses.
* The asset system can be used by command-line tools and web tools alike.
* Existing tools should be simple to adapt; if a tool takes a list of
page names, and language/project information, then using asset IDs as
an alternative source should be straightforward.
* A pipeline of tools could be started asynchronously, and their
progress could be tracked via JavaScript; once a tool has finished,
the next one in the chain could be run, all from the user's browser.

The main web API, and documentation page, is here:

http://toolserver.org/~magnus/wpipe/

That page also links to a generic "asset browser" :
http://toolserver.org/~magnus/wpipe/asset_info.php


As an appetizer, and feasibility demo, I adapted my own "CatScan
rewrite" to use assets as an optional output. This can be done by
copying the normal "CatScan rewrite" URL parameters and pasting them
here:

http://toolserver.org/~magnus/wpipe/toolwrap.php

This is intended as a generic starter page for command-line-enabled
tools. At the moment, only "CatScan rewrite" is available, but I plan
to add others. If you have tools you would like to add, I'll be happy
to help you set that up.
Right now, a tool is started by this page via "nohup &"; that could
change to the job submission system, if that's possible from the web
servers, but right now it seems overly complicated (runtime
estimation? memory estimation? sql server access? whatnot)
The web page then returns the reserved output asset ID, while the
actual tool is running; another tool could thus be "watching"
asynchronously, by pulling the status every few seconds.


Of course, this whole shebang doesn't make sense unless others are
willing to join in, with work on this core, or at least by enabling
some of their tools; so please, if you are even slightly interested in
a generic data exchange mechanism between tools, potentially leading
to a pipeline-able ecosystem, by all means step forward!

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver user phe account expired

[Magnus Manske]

On Sat, May 26, 2012 at 8:47 AM, Danny B.  wrote:
>> Perhaps we may be able to make it a cross-tool, which can either be
>> embedded in MediaWiki or standalone (I'm not aware of any  extension
>> doing that, but it's something I have considered several times myself).
>
> CategoryTree was/is sort of that...

So was GeoHack.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver user phe account expired

[Magnus Manske]

On Tue, May 22, 2012 at 6:16 PM, DaB.  wrote:

> In my eyes the toolserver developed behind this original idea and is now a
> good place to host stuff even when it is done

Just like Wikipedia was to Nupedia. Symmetry! :-)

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver user phe account expired

[Magnus Manske]

On Tue, May 22, 2012 at 1:30 PM, Platonides  wrote:
> On 22/05/12 11:22, Merlijn van Deen wrote:
>> How do we make sure tools do not disappear? By making multi-maintainer
>> projects for them. However, we see that this doesn't happen enough -
>> see also the thread about the expiration of soxred93's account.
>>
>> Options for improvement:
>>   - better communication with wikis - which tools are used a lot and
>> *thus* should be moved to mmp's?
>>   - easier creation of mmp's? I can imaging people don't move their
>> tools because it takes time to organise everything.
>
> It's relatively hard to create a MMP. Compare that with the complexity
> of doing a mkdir for creating a project in your account.
> Add to that the relatively low interest of other people for maintaining
> external projects (as shown by Magnus mail). There's little reason to
> create a MMP in advance.
> Plus, each of is coding using different languages, conventions and
> "frameworks" (helper functions).
>
> Maybe we should use a model where stable tools are available in a
> repository where all users can commit. The code can only be updated
> through that.
> As an alternative, each project could be in either open-gate or
> closed-gate model. In the first one, anyone can commit there. In the
> second one, there's just a subset of users which can directly commit
> (commits by others must be approved by a project member).
> If the accounts for all the project members expire (it gets orphan), the
> tool automatically changes to open-gate mode.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

An other idea, albeit one requiring more planning, coding, and
cooperation (and we are notoriously bad at two of these) would be to
separate front-end and back-end. If we could set up a MMP that
presents an API for the queries and data storage of several tools,
anyone (at the very least, anyone on the toolserver) could write a
front-end; also, taking over a front-end from someone else might be
more maintenance-friendly.

We could even offer consistent stylesheets for toolserver tools (hey,
one can dream?)

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver user phe account expired

[Magnus Manske]

On Tue, May 22, 2012 at 10:22 AM, Merlijn van Deen  wrote:
> On 21 May 2012 23:52, Lars Aronsson  wrote:
>> This madness has to stop. Wikisource depends on this service.
>> I understand the German toolserver is beyond repair, but
>> phe's statistics scripts should be moved to proper WMF servers.
>
> This madness has to stop. Wikis should stop depending on services that
> are not hosted in multi-maintainer projects!

I don't think that's realistic. There is barely enough of us
(volunteer programmers) as it is; co-maintaining someone else's code
would add a significant workload.

I have repeatedly offered all my tools for MMPs - so far, only
CommonsHelper2 and geohack are MMPs.

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Wikitech-l] 403: User account expired toolserver.org/~soxred93

[Magnus Manske]

On Wed, Mar 14, 2012 at 9:07 AM, Dr. Trigon  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 13.03.2012 17:05, Tim Landscheidt wrote:
>> Even if only for PHP there would be a critical mass, it'd be still
>> useful to Perl, Python and other developers.  For example, I think
>> it is a common problem to normalize links - e. g., are
>> "[[Diskussion:ABC_abc#.C3.A4.C3.B6.C3.BC]]" and "[[de:Talk:aBC
>> abc#äöü]]" pointing at the same resource? Most developers start
>> with the easy bits and end up with something that works for most of
>> their use cases, but fails if that line is overstepped (for example
>> "Image:" -> "File:").  If there was an existing module for this,
>> you wouldn't have to think about all the fringe cases yourself, but
>> could base upon the sweat poured by others :-).  If Me- diaWiki got
>> updated, you would just have to look at the changes in the PHP
>> module and port them to your language of choice.
>
> That is the reason why I occasionally use pywikipedia framework
> with toolserver tools also... ;) So use the pywikipedia as starting
> point for such a python library.
>
> +1
>
> Last thing I remember to miss (nice to have) was the toolserver
> notice [1] or a module to check if a given URL is safe (e.g.
> no "file:///etc/passwd")
>
> [1] https://wiki.toolserver.org/view/Toolserver_notice

At the risk of over-designing, would it be worth to gather
language-independent requirements for a module/library, on the
toolserver wiki or on meta, and then keep implementations of that
standard (yes, I know, "one more standard") available on the
toolserver? At the very least, a language-neutral brainstorming might
prevent design flaws, especially with database-with-API-fallback in
mind.

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] External authentication?

[Magnus Manske]

On Thu, Feb 9, 2012 at 1:08 PM, Victor Vasiliev  wrote:
> On Thu, Feb 9, 2012 at 3:37 PM, Magnus Manske
>  wrote:
>> Hi all,
>>
>> on my talk page, [[User:Pathoschild]] raised the idea of allowing
>> OpenID authentication to operate toolserver tools that currently rely
>> on TUSC. While I'd rather go for browserID [1] (not mutually
>> exclusive), it raised the point of which authentication is "good
>> enough" for using some toolserver tools, especially those that edit or
>> upload on Wiki(m|p)edia projects.
>>
>> Would these non-TUSC accounts need to be linked to Wiki(m|p)edia user
>> names? If so, how would this be done? If Wiki(m|p)edia were to provide
>> openID/browserID authentication, it would be a non-issue, but as it
>> stands, this would need to be done on the toolserver in some form,
>> which would most likely be more cumbersome than the current TUSC
>> account creation.
>>
>> Ideas?
>>
>>
>> Magnus
>>
>
> Well, Wikimedia should become OpenID/OAuth provider for that.

So, no change before 2020 then? ;-)

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] External authentication?

[Magnus Manske]

Hi all,

on my talk page, [[User:Pathoschild]] raised the idea of allowing
OpenID authentication to operate toolserver tools that currently rely
on TUSC. While I'd rather go for browserID [1] (not mutually
exclusive), it raised the point of which authentication is "good
enough" for using some toolserver tools, especially those that edit or
upload on Wiki(m|p)edia projects.

Would these non-TUSC accounts need to be linked to Wiki(m|p)edia user
names? If so, how would this be done? If Wiki(m|p)edia were to provide
openID/browserID authentication, it would be a non-issue, but as it
stands, this would need to be done on the toolserver in some form,
which would most likely be more cumbersome than the current TUSC
account creation.

Ideas?


Magnus


[1] https://browserid.org/

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] DNS-problems

[Magnus Manske]

Page loads fine in the UK, but I am quite familiar with the 502 error,
which, on certain days, shows up on every second request or so. Might
be coincidence, though.

Magnus



On Wed, Nov 30, 2011 at 6:02 PM, AleXXw  wrote:
> Hi!
>
> Right now: 502 Bad Gateway, when I'm in I get a SQL-Error 28
> (http://toolserver.org/~alexxw/Denkmalliste/index.php).
>
> Location: Austria, Provider: Kabelsignal
>
> Same Problem?
>
> Greets Alex
>
>
> On Wed, Nov 30, 2011 at 18:39, DaB.  wrote:
>>
>> Hello all,
>>
>> a part of the german community (all member of the same big ISP as far as I
>> see) reported to me problems with the DNS of the toolserver. I doubt that
>> the
>> problems are on our end, but just to be sure:
>> Have had anyone problems to reach the toolserver in the time since 20.
>> Nov.
>> until now, that are related to DNS (and is not a customer by german
>> vodafon or
>> arcor). I would be especialy interrested in non-germany-users.
>>
>> Sincerly,
>> DaB.
>>
>> --
>> Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] CrontabCr

[Magnus Manske]

On Fri, Nov 25, 2011 at 9:45 AM, John  wrote:
> Mine is betacommand

And one more (magnus):

Unable to run job: got no response from JSV script
"/sge62/default/common/jsv.sh".
Exiting.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Queues

[Magnus Manske]

On Wed, Nov 16, 2011 at 7:09 PM, Marlen Caemmerer
 wrote:
Hi,

> Which script does generate which error?

I got three messages on three days (it seems to have stopped now), all
like this:

Cron  cd /home/magnus/cronjobs ; ./all_most_wanted.pl

to magnus
Unable to run job: error: no suitable queues.
Exiting.
Unable to run job: error: no suitable queues.
Exiting.



It thus appears that
/home/magnus/cronjobs/all_most_wanted.pl
did not run on these days.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Queues

[Magnus Manske]

Thanks!

On Wed, Nov 16, 2011 at 9:13 AM, Marlen Caemmerer
 wrote:
> Hi Magnus,
>
> I guess you query these DBs:
>
>
> 'dewiki_p' , 'enwiki_p' , 'frwiki_p' , 'plwiki_p'
>
> dewiki and plwiki have not always been available until Monday so it is
> possible that your script failed for this reason.
> Everything should be back since yesterday so if there is still a problem
> please let me know.
>
> Cheers
>        nosy
>
> On Tue, 15 Nov 2011, Magnus Manske wrote:
>
>> Date: Tue, 15 Nov 2011 19:29:38
>> From: Magnus Manske 
>> Reply-To: toolserver-l@lists.wikimedia.org
>> To: toolserver-l@lists.wikimedia.org
>> Subject: Re: [Toolserver-l] Queues
>>
>> Hi nosy,
>>
>> first one (which I had missed) is from Nov 12, 5:15pm (UTC, I guess).
>> Seems to be a daily job.
>>
>> Mail title is: Cron  cd /home/magnus/cronjobs ;
>> ./all_most_wanted.pl
>>
>> Cheers,
>> Magnus
>>
>>
>> On Tue, Nov 15, 2011 at 5:15 PM, Marlen Caemmerer
>>  wrote:
>>>
>>> Hey Magnus,
>>>
>>> since when are they coming? Are they related to the databases s2 or s5?
>>>
>>> Cheers
>>>        nosy
>>>
>>> On Tue, 15 Nov 2011, Magnus Manske wrote:
>>>
>>>> Date: Tue, 15 Nov 2011 18:09:59
>>>> From: Magnus Manske 
>>>> Reply-To: toolserver-l@lists.wikimedia.org
>>>> To: toolserver-l@lists.wikimedia.org
>>>> Subject: [Toolserver-l] Queues
>>>>
>>>> I am now getting mails like this:
>>>>
>>>> Unable to run job: error: no suitable queues.
>>>> Exiting.
>>>>
>>>>
>>>> Haven't touched my cron settings for months...
>>>>
>>>> ___
>>>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>>>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>>>> Posting guidelines for this list:
>>>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>>>
>>>
>>>
>>> ___
>>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>>> Posting guidelines for this list:
>>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>>
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Queues

[Magnus Manske]

Hi nosy,

first one (which I had missed) is from Nov 12, 5:15pm (UTC, I guess).
Seems to be a daily job.

Mail title is: Cron  cd /home/magnus/cronjobs ;
./all_most_wanted.pl

Cheers,
Magnus


On Tue, Nov 15, 2011 at 5:15 PM, Marlen Caemmerer
 wrote:
> Hey Magnus,
>
> since when are they coming? Are they related to the databases s2 or s5?
>
> Cheers
>        nosy
>
> On Tue, 15 Nov 2011, Magnus Manske wrote:
>
>> Date: Tue, 15 Nov 2011 18:09:59
>> From: Magnus Manske 
>> Reply-To: toolserver-l@lists.wikimedia.org
>> To: toolserver-l@lists.wikimedia.org
>> Subject: [Toolserver-l] Queues
>>
>> I am now getting mails like this:
>>
>> Unable to run job: error: no suitable queues.
>> Exiting.
>>
>>
>> Haven't touched my cron settings for months...
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list: 
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Queues

[Magnus Manske]

I am now getting mails like this:

Unable to run job: error: no suitable queues.
Exiting.


Haven't touched my cron settings for months...

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] performance issues

[Magnus Manske]

On Sat, Oct 29, 2011 at 2:18 PM, DaB.  wrote:
> Hello,
> At Saturday 29 October 2011 15:04:30 DaB. wrote:
>> This queries, and others like it, have not caused problems for the
>> last one (two? I can't remember) years they've been in place.
>> Something has changed for the far, far worse.
>
> have you ever thought about the possiblity that maybe there are more users on
> the TS and more people who use the toolserver now than 2 years ago?

Unless there was an enormous influx of users in the last 12 hours or
so, this is not really an explanation.

> Or maybe
> it is just people like you, who let a query for a WEBTOOL run for 71 minutes!

If you look at the rather simple query, you'll see that it's not so
much that my tool needs 71 minutes to run. The issue is rather that
the database server takes 71 minutes for a query that shouldn't even
take 71 seconds.

> We are just short on hardware at the moment, and the situation will not change
> for some time, so we all have to live with the things we have and if a tool
> breaks because it consumes too much resources, then the dev. has to look if
> he/she can optimize it or it breaks!

The "work smarter, not harder" approach is neither helpful nor winning
many sympathies.

I am aware that the toolserver needs both hardware and admin
attention. I was not complaining about a lack of either. I was merely
commenting on a very recent change for the worse, which might have an
unrelated cause (a disk filled up, some configuration changed), and
might thus have an easy fix. Yelling at people who mention a new and
acute issue is not the way forward.

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] performance issues

[Magnus Manske]

On Wed, Oct 26, 2011 at 10:18 PM, Platonides  wrote:
> Marlen Caemmerer wrote:
>> Rosemary (one of the enwiki-DB-Hosts) seems to bring the maximum of the I/O 
>> that is possible, disk graphs are clipping there.
>> In the MySQL traffic graph you can see there is clipping too.
>> Strange thing about this is that this phenomenon started in the middle of 
>> september. Can anyone remember any important change in this time?
>> That is not the case with thyme (other enwiki DB Host) so it'd be a question 
>> wether we could share the load better.
>> Also I enabled query caching and will see if this is useful in any way.
>
> Maybe there's some job/tool which started at that time which loaded
> rosemary so much?

In the last three hours, I got no less than 21 immensely useless mails
from the query killer. The following query, apparently, was killed
after running for a whooping 71 minutes:

SELECT /* SLOW_OK */ /* GLAMOROUS */
gil_wiki,gil_page_title,gil_page_namespace,gil_to from
globalimagelinks,page,categorylinks where gil_to=page_title and cl_to
="Images_from_the_National_Archives_and_Records_Administration" AND
page_id=cl_from AND page_namespace=6 AND gil_page_namespace=""

This queries, and others like it, have not caused problems for the
last one (two? I can't remember) years they've been in place.
Something has changed for the far, far worse.

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Toolserver down

[Magnus Manske]

"The server at www.toolserver.org is taking too long to respond."

Also, no ssh (I tried nightshade).


Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob failing - help?

[Magnus Manske]

On Thu, Sep 22, 2011 at 2:46 PM, DaB.  wrote:
> Hello,
> At Thursday 22 September 2011 15:38:51 Magnus wrote:
>> 2011/9/22 René Kijewski :
>> > Am Wed, 21 Sep 2011 20:48:58 -0400
>> >
>> > schrieb Hersfold :
>> >> I've gotten this failure message the last two time my bot has tried
>
>> >
>> > same here. My
>
>> me too
>
> let me guess: All of you runs your task at midnight (together with a douzned
> others ;-)), don't you?

Nope:
51 01 * * * cd /home/magnus ; \rm core public_html/core
public_html/*/core /tmp/XMLTMP.* 2>/dev/null
55 05 * * * cronsub -s update_dupes $HOME/update_dupes.pl
02 04 * * * cronsub -s import_bibisbn /home/magnus/cronjobs/import_bibisbn.pl
43 04 * * * cronsub -s sifter /home/magnus/cronjobs/sifter.sh
10 * * * * cronsub -s popular /home/magnus/cronjobs/update_popular.pl
40 * * * * cronsub -s popular /home/magnus/cronjobs/update_popular.pl
05 02 * * 2 cd /home/magnus ; php glam_update.php
04 03 * * * cd /home/magnus ; php dnb_link_update.php
20 2 1 * * cd /home/magnus ; ./non_en_stats.pl >> public_html/non_en_stats.txt
15 17 * * * cd /home/magnus/cronjobs ; ./all_most_wanted.pl

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob failing - help?

[Magnus Manske]

2011/9/22 René Kijewski :
> Am Wed, 21 Sep 2011 20:48:58 -0400
> schrieb Hersfold :
>
>> I've gotten this failure message the last two time my bot has tried
>> to run. Does any one know what might be causing this? I don't
>> recognize the script it mentions.
>>
>> Unable to run job: got no response from JSV script
>> "/sge62/default/common/jsv.sh". Exiting.
>
> Hi,
>
> same here. My scripts worked well for some month now, so I guess it's a
> new problem at server side.

me too

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] A new tool request - find a translator

[Magnus Manske]

There you go:
http://toolserver.org/~magnus/transparent.php

Magnus


On Mon, Sep 12, 2011 at 8:18 PM, Ryan Kaldari  wrote:
> Could someone write a tool that allows me to search for the most
> recently active users on a certain project who speak a certain
> language (or combination of languages) natively?
>
> The user language categories seem to exist on most projects, so it
> seem like you could use those:
> http://en.wikipedia.org/wiki/Category:User_ja-N
>
> Ryan Kaldari
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Better toolserver status?

[Magnus Manske]

On Wed, Aug 17, 2011 at 3:33 PM, Casey Brown  wrote:
> On Wed, Aug 17, 2011 at 10:20 AM, Daniel Schwen  wrote:
>> Sure, I was a bit sloppy with the terminology, but the message still
>> stands. Even in an emergency, please think about _priorities_. Five
>> minutes taken to write a notification
>
> I think this needs a little bit of clarifying.
>
> A notification should be written if the downtime is going to be more
> extended or if you stop working on it for some amount of time (you
> can't do it, you go to bed, or something like that). If you're busy
> working on fixing it, that's your first priority and I don't think
> anyone wants you to stop prioritizing that. :-)

100% agree.

> Wikimedia ops are the same way -- it's the sysadmins first priority to
> fix up the techie stuff, that's their job. If someone's not doing
> anything at the moment and has some free time, *then* they should make
> communicating about the issue a priority. In the meantime, anyone
> watching what's going on in #wikimedia-toolserver can share what
> they're seeing with confused endusers.

Idea: How about some one-line message detailing current issues (if
any) with the toolserver that I could transclude from a single
location on top of my tools? That would alert all "my" users that
there is a problem with the toolserver (and not my tool specifically),
and that it's being worked on.

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Better toolserver status?

[Magnus Manske]

Several users were worried about toolserver tools that stopped working
without (to them) apparent reason. I have tried to explain in in
non-tech-German [1] and it seems that helped.

I was then asked if these issues are communicated somewhere centrally.
I found [2], which points to [3], which returns 404. It would probably
be too technical anyway. status.toolserver.org would not help the
average user either, even if it were always up-to-date.

So, should issues that affect the toolserver users (either the
code-writing or the tool-using, or both) be mentioned somewhere? There
is the toolserver blog [4], with the last entry from February 2010...

I think if all toolserver admins had write access to that blog, one of
them could spare 5 minutes (if that) to update the general public
about major issues or changes (changing stuff on the toolserver, even
supposedly minor changes, tends to break stuff...).

Cheers,
Magnus


[1] 
http://commons.wikimedia.org/wiki/Commons:Forum#Zentrale_Info_.C3.BCber_Technikprobleme
[2] http://meta.wikimedia.org/wiki/Toolserver/MaintenanceLog
[3] https://jira.toolserver.org/display/tech/Maintenance+log
[4] http://journal.toolserver.org/

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikimedia projects pageviews stats

[Magnus Manske]

On Thu, Aug 11, 2011 at 11:35 PM, RYU Cheol  wrote:
> Hi, folks!
> Pagecounts is available on toolserver database? I found Magnus have it for
> en and de in u_magnus_popular_p to
> serve http://toolserver.org/~magnus/toptopics.php.

These are just a tiny subset. The popular articles, as chance has it.

> If not, let me ask a question. Is it feasible to create one if we use the
> primary key with page_id rather than page_title?

Putting all that data into MySQL might strain the poor surviving DB
servers significantly.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Query-Killer is back in action

[Magnus Manske]

On Mon, Aug 15, 2011 at 11:39 PM, K. Peachey  wrote:
> On Tue, Aug 16, 2011 at 7:03 AM, Maarten Dammers  wrote:
>> Ah, you are the one who killed the categorization bot. Thanks for
>> announcing that beforehand (not!). It's very annoying that you suddenly
>> just decide to deploy a new toy the screws up our tools.
>>
>> Maarten
> It was re-enabling a service that has been around for a long time that
> [accidently] broke at one stage and because of the TS performance not
> being degraded was never really noticed or cared about.
>
> Recently a TS box went down and caused major issues (Replag was easily
> over 2 days at one stage for those databases) for the backup box
> handling the effected databases, DaB was monitoring this by hand and
> manually killing the processes to keep this somewhat saner then
> re-wrote and brought the tool back online to handle this when they
> noticed it was [the tool] was not functioning.
>
> This was a tool that was already meant to be running but by [your]
> "luck" wasn't, and was already documented on wiki as DaB pointed out
> over a year ago.
>
> Perhaps you should consider apologizing to DaB in regards to the tone
> of your email.

While it's wrong do insult DaB (or anyone here :-) I feel the
annoyance. It doesn't really matter if it was "supposed to" be running
but wasn't for some reason we mere mortals can't comprehend - queries
were fine, and then they suddenly they were not. I understand there
are reasons behind this (hardware failure, miser mode), but they
weren't obvious from the original mail.

So now I (and others, I suspect) are inserting /* SLOW_OK */ into
queries. That doesn't make the queries less or faster, but it returns
service to normal (for the tool writers, and the tool users). German
speakers might be familiar with the phrase "wenn's denn der
Wahrheitsfindung dient..."

Should we (the tool authors) make our queries faster? Absolutely. Can
we? Sometimes, maybe. Usually not. That might be due to personal
qualification, or system constraints. The local version of the "work
smarter, not harder" mantra is unlikely to improve the situation.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Query-Killer is back in action

[Magnus Manske]

On Mon, Aug 15, 2011 at 1:30 AM, DeltaQuad Wikipedia
 wrote:
> Quick question, is "Unable to run job: got no response from JSV script
> "/sge62/default/common/jsv.sh" " The killer? becuase ya, I got a lot of
> them, came home after about 5 hours to about 8 of them, and my API queries
> are quick...I don't get (unless it's the server) what i'm doing wrong.

It's not 'the killer', but it might be related, because I got those as
well, first time ever:

Unable to run job: got no response from JSV script
"/sge62/default/common/jsv.sh".

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Query-Killer is back in action

[Magnus Manske]

On Sun, Aug 14, 2011 at 8:12 PM, DaB.  wrote:
> At Sunday 14 August 2011 21:10:26 DaB. wrote:
>> Does anybody have a good suggestion what wiki to use for s3?
>> dawiki is the largest wikipedia on it, but I think it will have very
>> little edits during the night in Europe.
>
> I use dawiki and the hightest replag in the last 24h was 1978s. That should be
> more or less correct.

Update: I seem to be getting the vast majority of mails about
"z-dat-s4-a". Replag is <1000s but it seems strange that that one
shows up so predominantly.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Query-Killer is back in action

[Magnus Manske]

On Sun, Aug 14, 2011 at 5:42 PM, DaB.  wrote:
> Hello all,
>
> during the last week I kept an eye on the database-servers and killed queries
> which had run for too long. In the past we had a programm for that, but
> somewhen it stoped working and nobody fixed it AFAIK.
>
> Because watching the mytop is a boring thing and I have to sleep too, I used
> the last 2-3 days to write a substitute of the old query-kiler. This program
> is working now. It does more or less the same as I did, but it send you an
> eMail when it killed one of your query. I think that is an improvment and I
> can spend my time with other things.
>
> You can find the parameters of when and if the killer works at [1]. Of corse
> there is always the possiblity of adjustments. Feel free to open bugs at JIRA
> or discuss on teh mailinglist.

So the max run-time is dependent on replag. But, considering this:
Replag s3   6050h 10m 44s

I currently get bombarded with killed-query-mails :-(

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver status

[Magnus Manske]

On Tue, Aug 9, 2011 at 11:32 AM, Krinkle  wrote:
> On Aug 9, 2011, at 10:20 AM, Magnus Manske wrote:
>
>> On Tue, Aug 9, 2011 at 1:39 AM, MZMcBride  wrote:
>>> K. Peachey wrote:
>>>> On Tue, Aug 9, 2011 at 10:28 AM, MZMcBride  wrote:
>>>>> * http://status.toolserver.org/ is broken
>>>> s/broken/hasn't been manually updated/
>>>
>>> No, it's broken. You can click "edit this page" and view the source. It was
>>> updated last week, but it currently takes any status and outputs "ok",
>>> regardless of user input.
>>
>> The changes are pending and need to be approved by an administrator.
>>
>> /me ducks
>>
>
> /me thought it was only editable by administrators...

I was talking about the secret cabal meta-admins...

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver status

[Magnus Manske]

On Tue, Aug 9, 2011 at 1:39 AM, MZMcBride  wrote:
> K. Peachey wrote:
>> On Tue, Aug 9, 2011 at 10:28 AM, MZMcBride  wrote:
>>> * http://status.toolserver.org/ is broken
>> s/broken/hasn't been manually updated/
>
> No, it's broken. You can click "edit this page" and view the source. It was
> updated last week, but it currently takes any status and outputs "ok",
> regardless of user input.

The changes are pending and need to be approved by an administrator.

/me ducks

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Toolserver offline?

[Magnus Manske]

Back for me as well.

On Mon, Aug 8, 2011 at 8:13 PM, Béria Lima  wrote:
> Jus tested and both works for me :D
> _
> Béria Lima
> (351) 925 171 484
>
> Imagine um mundo onde é dada a qualquer pessoa a possibilidade de ter livre
> acesso ao somatório de todo o conhecimento humano. É isso o que estamos a
> fazer.
>
>
> On 8 August 2011 19:15, Magnus Manske  wrote:
>>
>> Can't lon into nightshade or willow (ssh keeps asking for password).
>> http://toolserver.org/ does not responsive.
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Toolserver offline?

[Magnus Manske]

Can't lon into nightshade or willow (ssh keeps asking for password).
http://toolserver.org/ does not responsive.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Tue, Mar 1, 2011 at 12:49 AM, MZMcBride  wrote:
> Magnus Manske wrote:
>> I understand that, and I didn't say lfs was better (it has plenty of
>> its own quirks), or that SGE was a bad choice - it seems to run well
>> enough, all things considered. It's just that things like "SGE might
>> interpret comments in your scripts as commands and die" or "you can't
>> run qsub from within qsub" are types of behaviour that are unusual to
>> say the least, and the toolserver users are left to the mercy of these
>> issues (I'd call them bugs) without warning or documentation. (Yes,
>> there is a wiki pages that says it can interpret special comments in
>> your script. That's somewhat different from "it will reinterpret your
>> Perl script and fall over").
>
> Having learned some lessons today, you should write improved documentation
> on the wiki. :-)

I might, once I've recovered from said "learning" experience...

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Disambiguation

[Magnus Manske]

On Mon, Feb 28, 2011 at 7:57 PM, Stefan Kühn  wrote:
> Hmm,
>
> but I will get the templates with a script. And when I show:
>
> http://en.wikipedia.org/wiki/MediaWiki:Disambiguationspage
> http://de.wikipedia.org/wiki/MediaWiki:Disambiguationspage
> http://fr.wikipedia.org/wiki/MediaWiki:Disambiguationspage
>
> then I can't see an standard format in this pages.
>
> Is there an other possibility?

For each language:
select pl_title from pagelinks,page where
page_title="Disambiguationspage" and page_namespace=8 and
pl_from=page_id and pl_namespace=10;

seems to work...

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Solving problems with qsub with -b y option

[Magnus Manske]

On Mon, Feb 28, 2011 at 3:24 PM, Marcin Cieslak  wrote:
>>> Marcin Cieslak  wrote:
>>>> Magnus Manske  wrote:
>>> On Mon, Feb 28, 2011 at 2:28 PM, Marcin Cieslak  wrote:
>>>>>> Magnus Manske  wrote:
>>
>>>>>> cronsub update_dupes /home/magnus/update_dupes.pl
>>>>> qsub: Unknown option
>>>>>
>>>>> qsub: Unknown option
>>>>>
>>>>> qsub: Unknown option
>>>>
>>>> This is because you have in your script
>>>>
>>>> #$cmd = "./cron_sql.sh" unless -e $cmd ;
>>>>
>>>> and this is recognized by qsub(1) as the embedded qsub option.
>>>>
>>>> If you change it to:
>>>>
>>>> # $cmd =
>>>>
>>>> it works again.
>>>
>>> Thanks!
>>>
>>> Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo
>>> meaning from my scripts (which predate qsub availability) anyway? If
>>> that is desired for some cases, why is that activated by default? Is
>>> there at least a qsub option to turn it off? Will that option work
>>> with cronsub?
>>
>> To quote man qsub(1):
>
> There is another option that's quite useful:
>
> qsub -b y path-to-program

Thanks, very helpful. Do you know if cronsub supports this?

> This causes path-to-program (use full path like /usr/bin/something or
> path relative to your $HOME or it won't work) to be executed
> as a "binary" program. Your script will not be copied to a temporary
> directory on the SGE cluster and will be ran in place.

I believe that is default behaviour anyway now? I vaguely remember a
mail concerning that WRT Python.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Mon, Feb 28, 2011 at 3:24 PM, River Tarnell  wrote:
> In article ,
> Magnus Manske   wrote:
>> Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo
>> meaning from my scripts (which predate qsub availability) anyway?
>
> Because it means you can include directives in your script indicating
> how it should run:
>
> #! /bin/ksh
> #$ -o $HOME/mytool.out
> #$ -l h_rt=3:0:0
> #$ -t 1-10
> #$ -tc 3
> # rest of script...
>
> This is easier than having a separate script just to start that one.

And more complicated than passing these parameters to qsub directly.


>> Is there at least a qsub option to turn it off?
>
> Yes.

Thankfully, Marcin was actually helpful in answering this. Your
attempt to distract from the actual issues at hand by "literal
interpretation" has been duly noted.

>> In my "day job", I work with a large compute farm (lfs, ~10K cores),
>> so I'm more than happy to work with, and migrate existing cronjobs to,
>> this system, but it would help if it weren't a deliberate minefield...
>
> I didn't write SGE.  I considered several existing batch scheduling
> systems and SGE seemed to be the best/most useful.  Of course that
> doesn't mean every user will like every feature it has.

I understand that, and I didn't say lfs was better (it has plenty of
its own quirks), or that SGE was a bad choice - it seems to run well
enough, all things considered. It's just that things like "SGE might
interpret comments in your scripts as commands and die" or "you can't
run qsub from within qsub" are types of behaviour that are unusual to
say the least, and the toolserver users are left to the mercy of these
issues (I'd call them bugs) without warning or documentation. (Yes,
there is a wiki pages that says it can interpret special comments in
your script. That's somewhat different from "it will reinterpret your
Perl script and fall over").

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Mon, Feb 28, 2011 at 3:01 PM, Marcin Cieslak  wrote:
>>> Magnus Manske  wrote:
>> On Mon, Feb 28, 2011 at 11:43 AM, Ilmari Karonen  wrote:
>>> On 02/28/2011 01:00 PM, Magnus Manske wrote:
>>>>
>>>> OK, so I gave up on starting my perl script using cronsub, which seems
>>>> to work only if I give the perl script a parameter (WTF?)
>>>
>>> Any parameter?  I smell a line break encoding issue.
>>>
>>> Try making sure there are no carriage returns in your crontab or in any
>>> of the scripts involved.
>>
>> Well, since that happens also when I start cronsub fron the shell, I doubt 
>> it...
>
> It seems like qsub dumps core when executed from inside of the SGE job.
>
> /sge62/default/spool/wolfsbane/job_scripts/210756[3]: 1253 Segmentation 
> Fault(coredump)
>
> I have filed https://jira.toolserver.org/browse/TS-963 about this. Really 
> strange.

Maybe some safety mechanism to prevent infinite recursion gone bad?

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Mon, Feb 28, 2011 at 2:28 PM, Marcin Cieslak  wrote:
>>> Magnus Manske  wrote:
>> On Mon, Feb 28, 2011 at 11:37 AM, River Tarnell  wrote:
>>> In article ,
>>> Magnus Manske   wrote:
>>>> OK, so I gave up on starting my perl script using cronsub, which seems
>>>> to work only if I give the perl script a parameter (WTF?), and instead
>>>> altered it to be run manually, generating a shell script that could
>>>> then start via cronsub.
>>>
>>> This should not be the case.  Please provide the exact script and
>>> command line you used and the error message.
>>
>>> cronsub update_dupes /home/magnus/update_dupes.pl
>> qsub: Unknown option
>>
>> qsub: Unknown option
>>
>> qsub: Unknown option
>
> This is because you have in your script
>
> #$cmd = "./cron_sql.sh" unless -e $cmd ;
>
> and this is recognized by qsub(1) as the embedded qsub option.
>
> If you change it to:
>
> # $cmd =
>
> it works again.

Thanks!

Wow, what a mess. Why does cronsub/qsub try to divine hidden vodoo
meaning from my scripts (which predate qsub availability) anyway? If
that is desired for some cases, why is that activated by default? Is
there at least a qsub option to turn it off? Will that option work
with cronsub?

In my "day job", I work with a large compute farm (lfs, ~10K cores),
so I'm more than happy to work with, and migrate existing cronjobs to,
this system, but it would help if it weren't a deliberate minefield...

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Mon, Feb 28, 2011 at 1:59 PM, River Tarnell  wrote:
> In article ,
> Magnus Manske   wrote:
>> > cronsub update_dupes /home/magnus/update_dupes.pl
>> qsub: Unknown option
>>
>> qsub: Unknown option
>>
>> qsub: Unknown option
>
> What is the output from this?
>
> $ ksh -x /opt/local/bin/cronsub update_dupes /home/magnus/update_dupes.pl


+ . /sge62/default/common/settings.sh
+ SGE_ROOT=/sge62
+ export SGE_ROOT
+ + /sge62/util/arch
ARCH=sol-amd64
+ + /sge62/util/arch -m
DEFAULTMANPATH=/usr/share/man
+ + /sge62/util/arch -mt
MANTYPE=man
+ SGE_CELL=default
+ export SGE_CELL
+ SGE_CLUSTER_NAME=toolserver
+ export SGE_CLUSTER_NAME
+ unset SGE_QMASTER_PORT
+ unset SGE_EXECD_PORT
+ [ 
/sge62/man:/opt/ts/gnu/share/man:/opt/ts/share/man:/opt/SUNWspro/man:/usr/postgres/8.3/man:/opt/ts/mysql/5.1/share/man:/opt/ts/python/2.6/share/man:/opt/ts/perl/5.10/share/man:/opt/ts/ruby/1.8/share/man:/usr/share/man:/usr/sfw/share/man:/usr/cluster/man:/opt/jobserver/man
=  ]
+ 
MANPATH=/sge62/man:/sge62/man:/opt/ts/gnu/share/man:/opt/ts/share/man:/opt/SUNWspro/man:/usr/postgres/8.3/man:/opt/ts/mysql/5.1/share/man:/opt/ts/python/2.6/share/man:/opt/ts/perl/5.10/share/man:/opt/ts/ruby/1.8/share/man:/usr/share/man:/usr/sfw/share/man:/usr/cluster/man:/opt/jobserver/man
+ export MANPATH
+ 
PATH=/sge62/bin/sol-amd64:/sge62/bin/sol-amd64:/opt/local/bin:/opt/ts/gnu/bin:/opt/ts/bin:/opt/ts/mysql/5.1/bin:/opt/ts/perl/5.12/bin:/opt/ts/perl/5/bin:/opt/ts/python/2.7/bin:/opt/ts/php/5.3/bin:/opt/ts/ruby/1.8/bin:/opt/ts/mono/2.0/bin:/opt/ts/tcl/8.5/bin:/usr/ccs/bin:/sge62/bin/sol-amd64:/usr/bin:/usr/sbin:/usr/sfw/bin:/usr/postgres/8.3/bin:/usr/cluster/bin:/opt/jobserver/bin
+ export PATH
+ unset ARCH DEFAULTMANPATH MANTYPE
+ 
PATH=/usr/bin:/sge62/bin/sol-amd64:/sge62/bin/sol-amd64:/opt/local/bin:/opt/ts/gnu/bin:/opt/ts/bin:/opt/ts/mysql/5.1/bin:/opt/ts/perl/5.12/bin:/opt/ts/perl/5/bin:/opt/ts/python/2.7/bin:/opt/ts/php/5.3/bin:/opt/ts/ruby/1.8/bin:/opt/ts/mono/2.0/bin:/opt/ts/tcl/8.5/bin:/usr/ccs/bin:/sge62/bin/sol-amd64:/usr/bin:/usr/sbin:/usr/sfw/bin:/usr/postgres/8.3/bin:/usr/cluster/bin:/opt/jobserver/bin
+ sflag=
+ QUEUE=all.q
+ OPTS=
+ hrt=
+ getopts slh f
+ shift 0
+ [[ 2 -ge 1 ]]
+ JOBNAME=update_dupes
+ shift
+ [ -z  ]
+ [[ 1 -ge 1 ]]
+ SCRIPT=/home/magnus/update_dupes.pl
+ + mktemp
TMPF=/tmp/tmp.5jaiw2
+ chmod 700 /tmp/tmp.5jaiw2
+ trap rm -f /tmp/tmp.5jaiw2 0
+ cat
+ 1> /tmp/tmp.5jaiw2 0<<
#! /bin/sh
cd $HOME
exec "$@"
exit 1

+ grep ^#\$
+ 0< /home/magnus/update_dupes.pl 1>> /tmp/tmp.5jaiw2
+ grep Full jobname:[ \t]*update_dupes$
+ 1> /dev/null
+ qstat -r
+ qsub -q all.q -N update_dupes /tmp/tmp.5jaiw2 /home/magnus/update_dupes.pl
+ 1> /dev/null
qsub: Unknown option

qsub: Unknown option

qsub: Unknown option

+ rm -f /tmp/tmp.5jaiw2

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Mon, Feb 28, 2011 at 11:43 AM, Ilmari Karonen  wrote:
> On 02/28/2011 01:00 PM, Magnus Manske wrote:
>>
>> OK, so I gave up on starting my perl script using cronsub, which seems
>> to work only if I give the perl script a parameter (WTF?)
>
> Any parameter?  I smell a line break encoding issue.
>
> Try making sure there are no carriage returns in your crontab or in any
> of the scripts involved.

Well, since that happens also when I start cronsub fron the shell, I doubt it...

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Mon, Feb 28, 2011 at 11:37 AM, River Tarnell  wrote:
> In article ,
> Magnus Manske   wrote:
>> OK, so I gave up on starting my perl script using cronsub, which seems
>> to work only if I give the perl script a parameter (WTF?), and instead
>> altered it to be run manually, generating a shell script that could
>> then start via cronsub.
>
> This should not be the case.  Please provide the exact script and
> command line you used and the error message.

> cronsub update_dupes /home/magnus/update_dupes.pl
qsub: Unknown option

qsub: Unknown option

qsub: Unknown option

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

On Sun, Feb 27, 2011 at 1:10 PM, River Tarnell  wrote:
> In article ,
> Marcin Cieslak   wrote:
>> >> Magnus Manske  wrote:
>> > I have a wrapper script that starts a handful of jobs on the
>> > toolserver (via qsub). But, when I put that wrapper script as a
>> > "normal" cronjob, I get some error messages about qsub not found.
>> > Starting the wrapper via cronsub doesn't work either.
>
> You should first run cronsub from the command-line with the same
> arguments.  If that still doesn't work, post the entire command line and
> any output from it.
>
> Once you have it working from the command-line then it should be fine
> fromc ron.
>
>> Actually cronsub does a litle bit more than sourcing
>> /sge62/default/common/settings.sh file, so I don't know
>> why it's not working for you. Maybe you could share some output?
>
> cronsub makes it easier to submit longrun jobs, automatically sets the
> default output file, runs the script from $HOME and prevents duplicate
> jobs being submitted.  Otherwise it's identical to qsub.


OK, so I gave up on starting my perl script using cronsub, which seems
to work only if I give the perl script a parameter (WTF?), and instead
altered it to be run manually, generating a shell script that could
then start via cronsub.

Script ( /home/magnus/cronjobs/most_wanted_wrapper.sh ) :
#!/bin/ksh
qsub -N mw_dewiki_p -j Y -o
/home/magnus/cronjobs/update_most_wanted.dewiki_p.out -l sqlprocs-s5=1
/home/magnus/cronjobs/update_most_wanted.pl dewiki_p
qsub -N mw_enwiki_p -j Y -o
/home/magnus/cronjobs/update_most_wanted.enwiki_p.out -l sqlprocs-s1=1
/home/magnus/cronjobs/update_most_wanted.pl enwiki_p
qsub -N mw_frwiki_p -j Y -o
/home/magnus/cronjobs/update_most_wanted.frwiki_p.out -l sqlprocs-s6=1
/home/magnus/cronjobs/update_most_wanted.pl frwiki_p
qsub -N mw_plwiki_p -j Y -o
/home/magnus/cronjobs/update_most_wanted.plwiki_p.out -l sqlprocs-s2=1
/home/magnus/cronjobs/update_most_wanted.pl plwiki_p

Running this script, or any of its individual lines, directly from
shell, works absolutely fine. However, on

> cronsub mostwanted /home/magnus/cronjobs/most_wanted_wrapper.sh

I get (in the error file):
/home/magnus/cronjobs/most_wanted_wrapper.sh[2]: 7274 Segmentation
Fault(coredump)
/home/magnus/cronjobs/most_wanted_wrapper.sh[3]: 7289 Segmentation
Fault(coredump)
/home/magnus/cronjobs/most_wanted_wrapper.sh[4]: 7290 Segmentation
Fault(coredump)
/home/magnus/cronjobs/most_wanted_wrapper.sh[5]: 7298 Segmentation
Fault(coredump)

I tried replacing qsub with cronsub, only to be met with a flood of
errors relating to all the options it doesn't know, including the sql
slot reservation (I guess that's my punishment for trying to be a good
citizen...).

Removing basically all options, using only lines like this:
cronsub -s mw_dewiki_p /home/magnus/cronjobs/update_most_wanted.pl dewiki_p
in the wrapper script, again runs peachy from shell, but barfs
coredumps again when run via cronsub.


So, as far as I am concerned, it's broken.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Cronjob starting cronsub

[Magnus Manske]

Hi,

I have a wrapper script that starts a handful of jobs on the
toolserver (via qsub). But, when I put that wrapper script as a
"normal" cronjob, I get some error messages about qsub not found.
Starting the wrapper via cronsub doesn't work either.

Anyone: What do I have to put in my .environment file (or whatever) so
that my "normal" cronjob can start qsub/cronsub jobs?

Thanks,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Google Art Project ripping

[Magnus Manske]

On Sat, Feb 5, 2011 at 7:57 AM, Kolossos  wrote:
> You can make your own pictures if you have
> a 7 Gigapixel camera ;-)

I can make my own pictures if I have a 6 megapixel camera as well. I
just can't make /the same/ pictures as Google, at least not easily.

The question is if leeching a couple of 7 gigapixel images to Commons
is worth p*g off the GLAMs (and Google).

I think it's not, and the exchange on the cultural-partners mailing
list about this topic seems to agree with me. These few pictures could
set back any months (years even) of efforts for good collaborations
with the GLAMs.

Aaah, politics...

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Google Art Project ripping

[Magnus Manske]

I see "NPG, reloaded" coming our way...

Note that Commons already has plenty of images from at least some of
these museums, e.g.:
http://commons.wikimedia.org/wiki/Van_Gogh_Museum

For the rest, we could just ask them "now that Google has put images
online, can we too?", then go and take our own pictures.

Too radical? ;-)

Magnus


On Fri, Feb 4, 2011 at 7:15 AM, Beao at Toolserver.org
 wrote:
> Hello everybody! I've been working on a bash script to rip the high quality
> images from the Google Art Project.
> From what I understand, all depictions of the original works go under
> PD-Art, even though Google claims otherwise in their FAQ. What I'm wondering
> is whether anyone is interested in helping me rip all these images. There
> are a lot of images to rip, and each rip takes at least an hour. All you
> need is a GNU/Linux OS and some basic knowledge on using the terminal.
>
> --
> Beao
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Monthly pageviews

[Magnus Manske]

2010/11/17 Frédéric Schütz :
> Would text files (similar to the current page views, but summarised over
> each month) be ok ? I have a few scripts (some from Erik Zachte, and
> some of mine) that could be adapted to do this.
>
> It's not the most efficient way to do it if you need random access (the
> API from stats.grok.se is probably better for this), but it would still
> be quite straightforward to parse.

That would be great for me (though of course a database would be even
better). Some time ago I tried to unzip/sort/merge/zip all monthly
data, but it took too long and too many resources.

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Monthly pageviews

[Magnus Manske]

On Mon, Nov 15, 2010 at 10:35 PM, MZMcBride  wrote:
> Magnus Manske wrote:
>> I know there are lots'o'files for daily (hourly?) pageview stats on
>> the toolserver.
>>
>> Are there aggregated counts for the whole month? So I only have to
>> check 1 file instead of hundreds (the aggregated file would, of
>> course, be smaller than the concatenated hourly ones).
>> Or maybe even as a database? (Onecan dream...)
>>
>> If not, does anyone volunteer to generate them? They'd really help
>> with my GLAM tools, increase Wikimedia outreach etc.
>
> Pageview stats are still a mess and there's no centralized or clean
> database, as far as I'm aware. Henrik's tool (stats.grok.se) has an API you
> can hit for monthly stats: http://stats.grok.se/json/en/201006/Barack_Obama
>
> That's probably your best bet right now.

And that's what I'm doing, but I need to look for tens of thousands of
pages, and it's very slow, not to mention traffic.

> From what I understand, Wikimedia is devoting resources to setting up Open
> Web Analytics. The first test run is supposed to be this week, I think.

That sounds good. Was that announced anywhere?

Thanks,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Monthly pageviews

[Magnus Manske]

Hi,

I know there are lots'o'files for daily (hourly?) pageview stats on
the toolserver.

Are there aggregated counts for the whole month? So I only have to
check 1 file instead of hundreds (the aggregated file would, of
course, be smaller than the concatenated hourly ones).
Or maybe even as a database? (Onecan dream...)

If not, does anyone volunteer to generate them? They'd really help
with my GLAM tools, increase Wikimedia outreach etc.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Golem issues

[Magnus Manske]

On Wed, Mar 31, 2010 at 9:27 PM, Mashiah Davidson
 wrote:
>> * Build a graph of Wikipedia articles in the main namespace, with
>>  wikilinks as vertexes.  Since some pages are not reachable from other
>>  pages, this is actually N disconnected graphs.
>> * Remove all edges which refer to disambiguation pages, date pages, or
>>  lists
>> * Remove the graph which contains the main page
>> * Produce a list of all remaining graphs.
>>
>> Is that roughly correct?
>
> It is roughly correct description of one of Golem's processing stages.

Well, I just did the experiment for German Wikipedia, using page_id
pairs in a temporary, non-memory table. In my user database (on the
same server as dewiki_p):

mysql> create temporary table delinks ( pid1 INTEGER , pid2 INTEGER )
ENGINE=InnoDB ;

mysql> INSERT /* SLOW_OK */ INTO delinks ( pid1,pid2 ) select
p1.page_id AS pid1,p2.page_id AS pod2 from dewiki_p.page AS
p1,dewiki_p.page AS p2,dewiki_p.pagelinks WHERE pl_title=p2.page_title
and p2.page_namespace=0 and pl_namespace=0 and p1.page_id=pl_from and
p1.page_namespace=0 ;

Query OK, 34964160 rows affected (32 min 59.29 sec)
Records: 34964160  Duplicates: 0  Warnings: 0

So, 35 million link pairs between namespace-0 pages, created in 33
minutes (~1 million links per minute). That's not too bad for our #2
wikipedia, and seems perfectly managable.

Depending on your usage, now add indices and spices :-)

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Shutdown of stable

[Magnus Manske]

OK, are we ready to switch all templates from



to



?
Seems to work well enough.

I know that there will be a redirect put in place [hopefully!:-)] but
maybe we should switch over on the main wikipedias before the deadline
to shake loose any problems before we don't have the fallback option
anymore.

Cheers,
Magnus


On Thu, Feb 4, 2010 at 9:08 PM, River Tarnell
 wrote:
> Hi,
>
> The stable server will be shut down permanently in two weeks, i.e. on the
> 18th.  This follows a 4 month migration period for moving tools.  Any tools
> that have not yet migration will be inaccessible after that date.  (However,
> any remaining files will be backed up.)
>
> hyacinth (the current stable server) will be repurposed as a redundant/fast
> database server.
>
>     - river.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Reminder: move your projects off stable -Geohack

[Magnus Manske]

On Sat, Jan 2, 2010 at 4:38 PM, Platonides  wrote:
> Which extension is it?
> http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/geo/ ?

Actually, most of it seems to be in
http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/gis/

> The first step should be to make the scripts available before moving to
> anything.
> Kolossos says he can't access geohack-scripts,
> https://fisheye.toolserver.org/browse/geohack shows an empty svn
> folder... (*ahem*, https://jira.toolserver.org/browse/TS-187 )

Yes, this is a mess, but IMHO this is largely not our (geohack
maintainers) fault; rather, things kept/keep changing on the
toolserver, stable or not, and things are generally not "in sync". For
example,
https://svn.toolserver.org/svnroot/geohack/
has a version of geohack, but fisheye does not show it. I am not sure
how up-to-date this is, either; and I can't commit to (or even
developer-checkout) that repository, neither as magnus nor as geohack.

For now, I have put the current sources from stable into my own toolserver svn:
https://fisheye.toolserver.org/browse/Magnus/geohack

If anyone can update the "real" repository, and convince fisheye to
show them too, that'd be peachy :-)

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Reminder: move your projects off stable -Geohack

[Magnus Manske]

On Sat, Jan 2, 2010 at 1:22 PM, Tim Alder  wrote:
> Hello,
> perhaps this would be a good moment to transfer Geohack to a regular
> mediawiki-extension on the main servers.
>
> I hear this was also the plan of brion vibber before he go.
> Who we need to ask now for this?

I asked Brion on the Paris meeting and he said he'd have a look. But,
I didn't hear anything back, so I assume he didn't.

wikimedia.org doesn't list an official CTO, so for the time being, I
guess Tim Starling is the person to ask.

Last time I had a long look at the code, the extension part seemed to
be in good condition. One half is using it as a special page (just a
wrapper around the current code calls), and getting the wiki template
from the cache instead of the http rendering.

Compatability might have deteriorated since then, though.

> I don't know why but I have no access to geohack-scripts anymore.
> So I can't do anything.

I'm still travelling, and probably won't have both access and time to
work on it until end of next week.

I guess the way to go is to get
http://toolserver.org/~geohack/
operational as a stop-gap until someone finally gets around to sign
off the extension and install it on the live site.

It seems to work /almost/ correctly. Maybe not all settings were migrated:
http://toolserver.org/~geohack/geohack.php?pagename=Berlin¶ms=52_30_2_N_13_23_56_E_

Cheers,
Magnus


>
> Greetings Kolossos
>
>
> river wrote:
>> Hi,
>>
>> As announced two months ago, we are planning to EOL the stable server in
>> favour of multi-maintainer projects on the normal Toolserver.  The following
>> projects are still running on the stable server:
>>
>> * geohack
>> * wma
>> * delinker
>>
>> We intend to repurpose the hardware currently used for the stable server to
>> provide redundancy database replication, but this can't happen until all
>> projects have migrated off the stable server.
>>
>>     - river.
>>
>>
>>
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] More advanced watchlists

[Magnus Manske]

On Tue, Nov 3, 2009 at 5:13 PM, Lars Aronsson  wrote:
>
> ==Watch a category of articles==
> 2010 is election year in Sweden, so I want to keep an eye on all
> articles in category:Swedish politicians, in recursive levels.
>
> I now do this on a weekly basis, using CatScan,
> http://toolserver.org/~daniel/WikiSense/CategoryIntersect.php
>
> I set wiki = sv.wikipedia.org
> search in category = svenska politiker
> with depth = 7
> for changes in last 168 hours (= 7 x 24 = one week)
> save as CSV format
>
> I then filter the output using the UNIX command line
> awk '$1==0 {print $2}'|sort|uniq -c|sort -nr
>
> to get a listing of which articles got the most edits.
>
> It would be nice to intersect this search with category:Living
> people, so dead politicians are filtered out.
>
> Are there already any smarter tools to do this?

Sorted by date, with intersection:


Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Dahne down

[Magnus Manske]

On Mon, Oct 12, 2009 at 9:39 AM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> we've spoked to Sun about this issue and they will ship a replacement
> motherboard for daphne, which hopefully will arrive this week.  this is
> a fairly complicated part to replace so it might take a bit longer to
> get someone to do it.

And by then, it probably can't sync to s2 anymore, right?

Is there a spare available that could be used for the s2 replica in
the meantime?

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] sql maintenance, Monday 14th

[Magnus Manske]

On Mon, Sep 14, 2009 at 9:03 AM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> this maintenance is now finished and sql is available again.

nightshade doesn't seem to agree:

mag...@nightshade:~$ sql dewiki_p
ERROR 2003 (HY000): Can't connect to MySQL server on 'sql-toolserver' (111)
/usr/local/bin/sql: no such database "dewiki_p"

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Change onwer by multi-maintainer-project

[Magnus Manske]

On Wed, Sep 9, 2009 at 2:58 PM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Jan Luca:
>> How can I run become in a SFTP-Client?
>
> you need to upload the files as your own user then copy them to the
> project account.

Or become user, make sure everything's group-read/writable, then
operate the STFP client under your normal user name, which should be
in the shared tool group (type "group" on the shell, it show the
shared tool name among others).

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Stange Perl (?) problem

[Magnus Manske]

On Fri, Sep 4, 2009 at 12:07 PM, River Tarnell
 wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Marco Schuster:
>> River, can you please take a look at this?
>
> we have 5 Toolserver admins who are quite capable of investigating
> problems; not everything has to be done by me.

If it's nothing obvious (probably related to the Solaris switch), I'll
file a JIRA bug.

Cheers,
Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] Stange Perl (?) problem

[Magnus Manske]

While I was on vacation, my CommonsHelper stopped working for direct
upload. Here's what happens:

* CommonsHelper (PHP) downloads an image to /tmp
* CommonsHelper invokes upload bot written in Perl
* Perl dies:
Can't open file /tmp/chHHaikY-dir/dummy.jpg : No such file or
directory at /opt/ts/perl/5.10/lib/vendor_perl/5.10/LWP/UserAgent.pm
line 394

However, the file exists at that point in time (this one still does on
wolfsbane, feel free to check), and is readable for everyone.

Any idea what's going on?

Magnus

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikimedia Foundation Grants $40, 000 to the Toolserver

[Magnus Manske]

YES! Fantastic!



On Wed, Jul 29, 2009 at 2:53 PM, Daniel Kinzler wrote:
> Hi all.
>
> The Wikimedia Foundation has approved a grant of $40,000 towards improving the
> Toolserver’s reliability. With that money, we'll by three database servers, so
> we can keep two copies of each wiki database instead of just one. This should
> greatly improve the availability of up to date data in the wiki dbs.
>
> A blog post with some more details is at
> .
> You can find the original grant reqwuest at
> .
>
> -- daniel
>
> ___
> Toolserver-l mailing list
> Toolserver-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] multi-maintainer tools

[Magnus Manske]

On Mon, Jul 27, 2009 at 3:13 PM, River
Tarnell wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Daniel Kinzler:
>> Also, that queue is full of spam :/
>
> the first thing i do when going through it is bulk-delete the spam.  only 
> takes
> a couple of minutes.

Well, it is called ts-ADmins ...

/me ducks

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] multi-maintainer tools

[Magnus Manske]

On Mon, Jul 27, 2009 at 2:04 PM, DaB. wrote:
> Hello,
> Am Montag 27 Juli 2009 14:24:32 schrieb Magnus Manske:
>> And since April, noone could invest 5 secs to reply to these two
>> queries, even if it's "use JIRA" ... why?
>
> ok, it was my fault. Is now everybody happy?

I wasn't trying to assign blame, merely pointing out where
communications could be improved.

I've submitted JIRA requests, all is well now.

Cheers,
Magnus

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l