Re: [Toolserver-l] 403 User account expired hit list

[Daniel Schwen]

Any chance we can get shell access back to place redirects? Was it not
possible to deactivate tools (i.e. cron jobs, PHP, any HTTP, except
redirects, etc.) and leave shell access open?

On Tue, Jul 1, 2014 at 5:21 AM, Marlen Caemmerer
 wrote:
> Hey,
>
>
> On Tue, 1 Jul 2014, Peter Schlömer wrote:
>
>> On Tue, Jul 1, 2014 at 1:07 PM, Marlen Caemmerer
>>  wrote:
>>>
>>> On Tue, 1 Jul 2014, Marlen Caemmerer wrote:
>>>
>
> I am pretty sure I did all my redirects protocol-relative like that,
> and they worked as intended (e.g. for /~dapete/ime/). Did you already
> make this change? Because now all my redirects lead to the HTTP
> version.
>
>>>
>>> Dapete, I guess this a tool of yours - where does it reside on TS?
>>>
>>> 221.204.196.130 - - [01/Jul/2014:11:03:15 +] "GET
>>>
>>> //tools.wmflabs.org/vcat/catgraphRedirect?cat=El_Santo&d=10&format=png&lang=fr&links
>>> =wiki&sub=0&wiki=wikipedia HTTP/1.1" 404 1304 - 1292604356 2
>>
>>
>> The redirect is for /~dapete/catgraph/graph.php
>>
>
> Really strange. I reverted the changes for you htaccess and the URL redirect
> still works and gives a strange log entry, dont know why.
>
> Says No webservice btw...
>
> Anyways, seems I will revert all changes to htaccess files I made for now.
>
> Cheers
> nosy
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Do the problems persist?

[Daniel Schwen]

looks good to me

On Wed, Feb 5, 2014 at 9:56 AM, Silke Meyer  wrote:
> Hi all,
>
> Mette is trying to figure out if the problems persist. Do they?
> I can acces the tools via web again, I think since the moment nosy
> rebootet "something". ssh on willow and from there to nightshade /
> yarrow is working again.
>
> Can we call this issue closed or do you still experience problems?
>
> Silke
>
> --
> Silke Meyer
> Internes IT-Management und Projektmanagement Toolserver
>
> Wikimedia Deutschland e.V. | Tempelhofer Ufer 23-24 | 10963 Berlin
> Tel. (030) 219 158 260
>
> http://wikimedia.de
>
> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
> Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
> unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das
> Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Can not login to nightshade and willow via ssh

[Daniel Schwen]

There is a bit more wrong than just user-store. I was still logged in
to login.toolserver.org this morning but the filesystem wen away (home
dir). I closed the connection and tried to log back in without
success.

On Sat, Feb 1, 2014 at 1:34 PM, Tim Landscheidt  wrote:
> (anonymous) wrote:
>
>> I can't login to willow and nighsthade via ssh. They both show some welcome
>> text and nighshade ends up at showing the "Last login: ...", willow at "The
>> following screen sessions are active...". Nothing happens then, neither it
>> continues, nor the connection gets disconnected.
>
>> Anybody has the same issues?
>
> ssh to nightshade works again (compared to last night), but
> apparently hangs in /etc/profile.d/quota.sh.  Aborting that
> with ^C, I get a prompt.  "ls /mnt" there shows user-store,
> "ls -l /mnt" hangs in status D.  Accessing /mnt/user-store
> on yarrow shows no problems; /etc/fstab's entries are iden-
> tical on both hosts.
>
> So it looks to me as if there's a problem with the
> /mnt/user-store rosemary NFS mount on nightshade.
>
> Tim
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] TS and/vs. Labs

[Daniel Schwen]

I just don't get this discussion. We've been singing the same song for
years now: "Tools should be opensource and freely licensed to make
sure they are forkable, independent of single owners, and thus
durable"
Now people are suddenly complaining when opensource is enforced that
their closed source tools will die?!
Well, D'uh!

On Thu, Aug 29, 2013 at 3:21 PM, Jeremy Baron  wrote:
> Hi,
>
> Can you elaborate?
>
> On Thu, Aug 29, 2013 at 8:14 PM, Merlissimo  wrote:
>> Am 29.08.2013 21:09, schrieb Ryan Lane:
>>> Tools and bots are depended on by the movement to properly run. They're
>>> an extension of the infrastructure. The projects should be forkable and
>>> that includes the tools and bots that keep it running.
>>>
>>> You're arguing to keep parts of the infrastructure closed source. To
>>> what end? How many tools are currently using closed source?
>>
>> For example all my bots MerlBot, MerlLinkBot and MerlIwBot combining
>> about 80 different tools developed in the past 6 years.
>
> What makes these closed? Whose copyrights are the limiting factor
> here? (links to the pages for those things if they exist would be
> great. e.g. libraries)
>
>>> How hard would it be to modify them to use open source alternatives?
>>
>> I am estimating about 150-200 hours of work. I also have to solve the
>> problem that i know the source code of the used free but closed source
>> libary. So according the OTI i am not allowed to write an open source
>> replacement. The current agreed solution is that sb. from WMDE has to
>> rewrite some parts. I am currently thinking about how to minimize this
>> paid work.
>
> What's OTI?
>
> Please don't use free to mean "no cost" on this thread. In fact, it
> would be good to avoid "free" entirely. For "free software" use libre
> and for "free as in beer, no cost to use" use gratis.
>
> -Jeremy
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Patricia Pintilie

[Daniel Schwen]

At this point it doesn't even matter anymore. None of her posts so far
made any sense or contributed in a meaningful way. Please block.

On Thu, May 23, 2013 at 8:07 AM, Jeremy Baron  wrote:
> On May 23, 2013 7:30 AM, "Patricia Pintilie"  wrote:
>> General chat • Re: Navigating Multiboot GRUB2 menu entries successfully. 
>> http://forum.porteus.org/viewtopic.php?t=2195&p=15042#p15042
>
> Hi Patricia,
>
> Are you human? Please send me the square root of 16.
>
> Thanks!
>
> -Jeremy
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikidata tables

[Daniel Schwen]

> FWIW, I have implemented a query-able stand-alone web server that keeps all
> of the wikidata property-item-links in memory. This uses the wikidata dumps

That does not sound too terribly scalable. I did the same thing
(custom webserver, data kept in memory) for the map labels of my
WikiMiniAtlas, and had to abandon this in order to be able to support
more languages. And my data is only a subset of what I expect to show
up in WikiData.
On top of that not being able to join data from there with other DBs
is a serious deficiency. Same goes for the suggestion to "just use the
API".
Daniel

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikidata tables

[Daniel Schwen]

> if these JSON-data is stored where the normal wiki-text is, it is imposable
To my understanding it is.

> for us to replicate it: Because we have no access to these wmf-servers, there
IMO that was a questionable design decision. JSON plaintext storage in
SQL is shoehorning a do-it-yourself object store onto a classical
RDBMS.
Postgres at least has hstore. This may be even a genuine usecase for
one of those hipster databases (noSQL like mongdb etc.). But who knows
what points were taken into consideration when making this decision.

> would be no way to separate Wikidata from the rest
I don't understand why separating plaintext storage between different
projects would be an issue. Is it all lumped into one storage
"namespace"?
I'm sure nobody at Wikimedia would be the least bit motivated to make
this data available to the toolserver, but maybe it will be usable in
labs. Otherwise it would be quite a waste of a great opportunity.

> and/or we have not enough disc-space.
If you can separate it out I seriously doubt that wikidata would
require storage any where near as large in magnitude as the other
wikimedia projects (at least in the mid-term)

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Wikidata tables

[Daniel Schwen]

I can (barely) understand why wikitext is not available on the
toolserver, but the JSON data you are talking about does not seem
copyrightable and much lower in volume. The usefulness of the wikidata
mirror on the toolserver is rather low without the actual wikiDATA.
Daniel

On Thu, Apr 18, 2013 at 8:57 AM, Byrial Jensen  wrote:
> Den 18-04-2013 11:21, Lydia Pintscher skrev:
>
>> On Thu, Apr 18, 2013 at 9:52 AM, Magnus Manske
>>  wrote:
>>>
>>> Just wondering what the status of exposing all wikidata tables on the
>>> toolserver is.
>>>
>>> Currently, there are a few wb_* tables with item labels, descriptions,
>>> aliases, and language links.
>>>
>>> But the tables (whatever they are called) containing item-to-item
>>> connections appear to be missing. Maybe because they were added later?
>>
>>
>> As far as I know they're only saved in JSON where usually the article
>> text is stored and not in separate tables.
>
>
> You can see in the pagelinks table which properties and which items an item
> is connected to by statements, but not how the properties and items are
> paired together.
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Installing Exiv2: `Make` errors

[Daniel Schwen]

> From my previous email:
> I was actually trying to install gexiv2, the "later and greater" version of
> pyexiv2, but...hey, this'll work too.

Huh? What previous email? I saw no mention of that.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Installing Exiv2: `Make` errors

[Daniel Schwen]

Why are you trying to compile it yourself.
It is already available on the toolserver. I'm running a bot that uses
it (through pyexiv2)

On Thu, Mar 28, 2013 at 10:33 PM, Theopolisme  wrote:
> I'm trying to install the Exiv2 library (locally, for just me), but run into
> the following error:
>
> theo@willow:~/tools/exiv2-0.23$ make
> ... [some make jibber-jabber]
> /bin/sh: test: argument expected
> make[1]: *** [XMPMeta-GetSet.o] Error 1
> make[1]: Leaving directory `/home/theo/tools/exiv2-0.23/xmpsdk/src'
> make: *** [xmpsdk] Error 2
>
> It looks like this issue—'argument expected'—definitely has arisen before
> (example:
> [https://mail.gnome.org/archives/garnome-list/2006-June/msg00035.html]) but
> I couldn't find a solution. Any enlightenment?
>
> --
> Theo (User:Theopolisme on Wikipedia)
> http://enwp.org/wiki/User:Theopolisme
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [TS logo] Fwd: Free as in Wikimedia Foundation

[Daniel Schwen]

What a nonsense issue. This superfluous discussion is fueled by two
fallacies, the confusion of copyright and trademarking and an
unhealthy paranoia toward the foundation. Protection of logos
associated with the wikimedia community is a good thing. The
foundation is an asset to the community! They can offer legal support
in cases of abuse of wikimedia related symbology. It is absurd to
create a spectre of a community-suing evil foundation while at the
same tie ignoring the very real threat of dilution and abuse of
wikimedia symbols and resulting damage of wikimedia community
reputation by spam, phishing, link farming etc. sites.
Sorry, but this is alarmist hippie crap and typical "netizen-outrage".

On Tue, Mar 19, 2013 at 7:20 AM, Federico Leva (Nemo)
 wrote:
>
>
>
>  Messaggio originale 
> Oggetto: [Wikimedia-l] Free as in Wikimedia Foundation
> Data: Tue, 19 Mar 2013 13:12:04 +0100
> Mittente: Tomasz W. Kozłowski
> A: Wikimedia Mailing List
>
> Hi community,
> I would like to bring to your attention a matter that's currently
> being discussed on Meta, one that has not yet gained too much interest
> (though it was discussed during IRC office hours, and was mentioned on
> one mailing list, as far as I see).
>
> It seems that the Wikimedia Foundation registered a community logo
>  as
> a trademark in the United States, with the international application
> still pending.
>
> The logo was originally created in 2006 by User:WarX (Artur
> Fijałkowski) , and was adopted as the logo of Meta-Wiki in 2008 — and
> as far as I can recall, the very point of it being created was to (1)
> have a community logo released into the public domain and (2) to have
> a community logo which was /not a trademark/.
>
> I am especially worried about the WMF not informing the community
> about their trademark registration — we have only found out about it
> via an edit on Commons
> ,
> and then after asking about it during IRC office hours at the end of
> January.
>
> As far as I understand, the WMF has not discussed trademark
> registration with the author of the logo
> 
> — though obviously, since Artur-WarX released it into the public
> domain, it would've only be good manners, and not a legal requirement.
>
> The discussion is taking place on Meta at
> , and all
> comments are welcome.
>
> --
> Tomasz W. Kozłowski
> a.k.a. [[user:odder]]
>
> ___
> Wikimedia-l mailing list
> wikimedi...@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] rosemary's copy of enwiki_p also corrupt?

[Daniel Schwen]

>
> Yes, Asher is working on it this month. We've got most of the hardware
> provisioned. Ryan will work on user grants when he's back. No promises
> yet that we'll exceed our original deadline (which is launch of this
> functionality in the first quarter of 2013) but we'll do what we can
> to move it forward.
>

Any updates regarding the possibility to perform joins between wiki DBs and
user DBs? Or wikipedia DBs and the Commons database? As it surely has been
mentioned, this is a vital feature of the toolserver DB replication setup
and would be much appreciated (i.e. desperately needed) on labs.
Daniel
___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Reasons for not migrating to Tool Lab

[Daniel Schwen]

At the risk of outing myself as "naive": I do not see this as a
problem like MZMcBride does. I think the foundation should have earned
our trust by now and them locking down the data does not seem like a
credible threat to me.
In any case:

a) you can download dumps to access the data independently from WMF
b) the replication to the TS is already "at the mercy" of WMF. The TS
does not make the data any free-er.

Best,
Dschwen


> I think I'd add "general direction of centralizing everything under a single
> Wikimedia Foundation is a bad idea" as a permanent blocker. Maybe there's a
> reasonable case for why deprecating the Toolserver and creating Wikimedia
> Labs is a great idea, but I don't see it yet.
>
> I don't see why each (Wikimedia) chapter shouldn't have its own replica of
> the databases. We want free content to be free (and re-used and re-mixed and
> whatever else). If you're going to invest in infrastructure, I think it
> makes more sense to bolster replication support than try to compete with the
> Toolserver.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Future of the toolserver

[Daniel Schwen]

Another point to consider is that for some projects there is
considerable interdependence (this may not be the norm).
My WikiMiniAtlas for example depends on

1. the OSM database mirror being available
2. the WIWOSM project (although I could proxy that from the TS)
3. Dispensers coordinate extraction database (GHEL)

These dependencies will increase the viscosity of the migration process.
In theory database connections could be ssh tunneled although

1. that may not be a very stable or fast solution
2. someone/somewhat is blocking ssh traffic from labs to the
toolserver cluster (other ssh connections work fine) what is going on
here?!

Dschwen

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Web services are down

[Daniel Schwen]

wolfsbane http is down, ortelius is still working.

On Tue, Aug 28, 2012 at 8:13 PM, DeltaQuad  wrote:
> Down for me and a 2 other users on IRC.
>
> DeltaQuad
> English Wikipedia Administrator and CheckUser
>
> On 8/28/2012 10:10 PM, John wrote:
>>
>> works for me
>>
>> On Tue, Aug 28, 2012 at 10:01 PM, Hersfold  wrote:
>>>
>>> I'm getting a 502 Bad Gateway error when trying to access any of the
>>> toolserver websites, except for the wiki which appears to simply time
>>> out.
>>> status.toolserver.org says it's up, could someone check into this?
>>>
>>> --
>>> 
>>> User:Hersfold
>>> hersfoldw...@gmail.com
>>>
>>>
>>> ___
>>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>>> Posting guidelines for this list:
>>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] s1 replag update, suggestion, and question

[Daniel Schwen]

I'm a little confused as to which DB server we are talking about. I
need access to

enwiki-p.db.toolserver.org
hap-s1-user.esi.toolserver.org.

is that sql-s1-user or sql-s1-rr or what?

Daniel

On Wed, Aug 8, 2012 at 7:46 AM, Russell Blau  wrote:
> (TL;DR? Skip down three paragraphs to the possible workaround)  Last
> month, I reported on the progress of SHA-1 updates from the WMF servers,
> and noted that s1 replag was likely to continue to be a problem for a
> number of weeks.  As I said then, the WMF was using (at least) three
> processes to populate the SHA-1 field on three separate blocks of
> revision records.  All these changes then were being replicated to the
> Toolserver's copies of the databases, and this flood of updates was
> causing the replag.
>
> The three blocks were being populated at different rates (for reasons
> that are beyond my knowledge). On July 23 at about 15:00 UTC, rosemary
> (sql-s1-rr) completed updating the first of the three blocks. The other
> blocks continued to be populated (and at some point the WMF started
> another process to help finish off the slowest block), but the rate of
> updates was somewhat less, and rosemary actually caught up on its
> backlog and reached zero replag within about a day after this milestone.
>
> The situation on thyme (sql-s1-user) is less favorable, as we all know.
> The replag on that server got much higher to start with, and thyme
> didn't even reach the end of the first block until Sunday August 5 at
> about 12:00 UTC. Unlike the situation with rosemary, the reduced load
> after this event did not make any noticeable difference to the replag,
> which has continued to increase for the past three days at much the same
> rate as before.  The next milestone will be completion of the second
> major block, which looks like it will occur either late on Friday August
> 9 or early on Saturday August 10 UTC, barring any other major problems
> (like the WMF server outage on Monday which caused replication at the TS
> end to stop for several hours).  At that point, the load from SHA-1
> updates should be roughly about 30% of what it had been during July. One
> would think that would allow the replag to drop, but since the events of
> this week, I can't be confident of that.
>
> There is a possible workaround.  The TS could treat this like a server
> outage; copy user databases from thyme to rosemary and then point
> sql-s1-user to rosemary, which currently has no replag. Rosemary would
> then have to handle twice the load, but thyme should start to recover
> very quickly with no user-generated queries hitting it. Once thyme has
> recovered, point sql-s1-rr to it.
>
> Downsides: (1) this would require several hours of downtime for
> sql-s1-user while the user databases are copied; all tools that require
> access to user databases would be offline entirely for this period. (2)
> it would have to wait until our volunteer TS admins have time to do it.
> (3) the added load on rosemary could cause replag to grow there,
> although I doubt it would come anywhere near the 14+ days replag we are
> dealing with now on thyme. (4) this could all be unnecessary since thyme
> might recover on its own once the SHA-1 update load is reduced, although
> I don't know any way of forecasting that and experience so far has not
> been encouraging.
>
> Question for those of you who operate and/or use tools that access s1
> (enwiki):  would you be willing to accept several hours of service
> outage and the other downsides in exchange for getting rid of the 14-day
> replag?
> --
>   Russell Blau
>   russb...@imapmail.org
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list: 
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Fairness on the toolserver

[Daniel Schwen]

There have been arrangements with some projects that need a lot of
space by nature, like mine, The WikiMiniAtlas. But loging in today for
the first time I received a quota warning with an ultimatum to free
space within 4.9 days. Since when is that active?
Daniel

On Tue, May 15, 2012 at 6:32 PM, Hersfold  wrote:
> I'd prefer reasonably high "hard" limits - if someone does need more space,
> they can request it from the sysadmins. Quite frankly, if people are
> over-using resources intentionally now, there's nothing to say they won't
> continue to do so in future given the chance.
>
> 
> User:Hersfold
> hersfoldw...@gmail.com
>
>
>
> On 5/15/2012 6:38 PM, Marcin Cieslak wrote:

 Krinkle  wrote:
>>>
>>> On May 14, 2012, at 11:31 PM, DaB. wrote:
>>>
 Hello all,

 the resources on the TS are free, but limited; so we all have to
 use the resources fair. Some limits (like memory-usage) are set
 and controlled by the system, but others are not and it is in the
 responsibility of every single user to make sure to not mis- or
 overuse resources.  So it is for example NOT a good idea to run
 200 processes in parallel to get more CPU-resources than you would
 normally get. And it is not a good idea to use a amount of memory
 which is just below the slayer-daemon-limit without any purpose.
>>>
>>> It would only be reached if: * The user might accidentally run
>>> a proces that is taking more than he expected. The limit would
>>> kill the problem before it runs out of hand. * The user might be
>>> misbehaving. The limit is working against that. * The user might
>>> be hosting a tool that naturally takes more space. He requests the
>>> additional space and explains why he needs it. In most cases this will
>>> be granted without any problems.
>>
>> UNIX has a good infrastructure to do that already. See limit(1),
>> setrlimit(2).  Actually Solaris (in comparison to Linux) is much more
>> flexible in working
>> with resource assignement for projects.
>>
>> What could be done is to set "soft" memory, number of processes, maximum
>> running time of a process and other limits by default.
>>
>> Users, if there is a legitimate need, could raise those limits themselves
>> (like, "I need 2GB of RAM for my PHP process to run MediaWiki unittests
>> now").
>>
>> Right now I don't see any limits on willow:
>>
>> willow$ ulimit -a
>> time(seconds)        unlimited
>> file(blocks)         unlimited
>> data(kbytes)         unlimited
>> stack(kbytes)        10240
>> coredump(blocks)     unlimited
>> nofiles(descriptors) 256
>> vmemory(kbytes)      unlimited
>>
>> If this would be abused, "hard" limits can be introduced, which cannot be
>> overriden by user.
>>
>> Also, system accounting could also be enabled (see sar(1), sar(1M))
>> to regularly report on all resources used by users in the system.
>>
>> Process accounting (see acct(1M)) also provides a valuable
>> information about what was run by users and when.
>>
>> Solaris has even ready cron scripts to prepare daily reports
>> to summarize the information and send them by email.
>>
>> //Saper
>>
>>
>> ___
>> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
>> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>> Posting guidelines for this list:
>> https://wiki.toolserver.org/view/Mailing_list_etiquette
>
>
> ___
> Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
> Posting guidelines for this list:
> https://wiki.toolserver.org/view/Mailing_list_etiquette

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Better toolserver status?

[Daniel Schwen]

> I think this needs a little bit of clarifying.

Ok, here we go:

Priority should _not_ be whether "the toolserver" is running, it
should be "are the tools running"!

I thought this was a no-brainer.
You may think these are the same thing, but it is not! Finxing a
toolserver problem is the necessary _minimum_, but it is not
_sufficient_ to assist in making sure the tools run. For that you need
to keep the tool developers in the loop!

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Better toolserver status?

[Daniel Schwen]

> A maintaince is something you know it will happen BEFORE it happens. An
> outtage/crash/emergency is something you have no idea it will happen in the
> next minute.

Sure, I was a bit sloppy with the terminology, but the message still
stands. Even in an emergency, please think about _priorities_. Five
minutes taken to write a notification could save several tool
developers hours of
* figuring out what is happening
* having long downtimes due to crashed processes, broken/inconsistent
database tables
* cleaning up after bots that break or go berserk due to server outages

Sure, you can start pointing fingers at the tool developers and yell
"write better tools". But let me remind you (in case it was
forgotten). The tool developers already donate their free time to
create tools that benefit the wikimedia userbase. We are not doing
this purely for our amusement. I'm grateful the toolserver exists, but
I'm also doing my share of the work! All I'm asking is that the admins
recognize that part of their service should be not making my work
harder than it already is.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Better toolserver status?

[Daniel Schwen]

> announce. I think it is clear that we than have other things to do then
> writing long texts ;-).

No, that is not clear _at all_. I realize the importance of toolserver
maintenance. But let's face it, you guys are providing the
infrastructure FOR THE TOOLS. No enduser cares about the toolserver
itself, people care about whether the tools are working or not. And a
manytimes these tools break or need intervention when a part of the ts
cluster breaks. The tool authors need to be informed so that they in
turn can take action to fix their tools. This is important, and I
think it warrants taking a few minutes to withe a notification.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] db servers s6 and s7 are apparently read-only right now

[Daniel Schwen]

> done with https://jira.toolserver.org/browse/MNT-1065. Please be carefull with
> writes for the beginn. Please also remember that your databases, which were

What does "careful with writes" mean?
INSERT INTO u_dschwen.test ('hallo') WITH CARE PLEASE;

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] db servers s6 and s7 are apparently read-only right now

[Daniel Schwen]

And they will be switched back tomorrow, as I just heard on IRC.
I'm just gonna post this here, as I assume it is not really supposed
to stay a secret...

On Thu, Aug 11, 2011 at 2:03 PM, Daniel Schwen  wrote:
> http://status.toolserver.org/ does not show this, and I don't see an
> announcement.So maybe this is new and unknown.
>

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

[Toolserver-l] db servers s6 and s7 are apparently read-only right now

[Daniel Schwen]

http://status.toolserver.org/ does not show this, and I don't see an
announcement.So maybe this is new and unknown.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] General maintenance notice: Monday, June 6th

[Daniel Schwen]

dschwen@nightshade:~/dschwen_bot$ python gps_exif_bot2.py
/home/dschwen
Traceback (most recent call last):
 File "gps_exif_bot2.py", line 9, in 
   import pyexiv2
ImportError: No module named pyexiv2

Still unavailable. I opened a ticked (TS-1066) a few days ago, no
reaction. I would install this by hand if the dependencies weren't so
inconvienient (boost::python, exiv2)

This kills my bot that automatically extracts GPS data and inserts
location templates for images on commons. People seem to be missing
that service.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] [Toolserver-announce] General maintenance notice: Monday, June 6th

[Daniel Schwen]

> * The Python module "pyexiv2" is currently unavailable.  This will be
>  fixed tomorrow (7th).  (Sorry.)

dschwen@nightshade:~/dschwen_bot$ python gps_exif_bot2.py
/home/dschwen
Traceback (most recent call last):
  File "gps_exif_bot2.py", line 9, in 
import pyexiv2
ImportError: No module named pyexiv2

Still unavailable.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Privacy Violation?

[Daniel Schwen]

You could instead just output a correlation-factor for the two
edittime curves (integrate the product for example). That would reduce
the amount visible data drastically, making it very hard to tell
anything about the editing habits of each individual.
Daniel

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Privacy Violation?

[Daniel Schwen]

> https://encrypted.google.com/search?q=This+message+and+it%27s+attachments+ma
> y+contain+confidential+information+&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:e
> n-GB:official&client=firefox-a

>From the very first Google search result:

"Email disclaimers are statements you often find appended to the end
of an email.[..] What they have in common is that my blood starts
boiling at their sight. [..]
While these statements have a sane purpose, namely to avoid liability
for the content of the emails which you cannot control, it is
interesting that so many of them are... well... misguided. The best
examples are stating two things. For one, you are bound to certain
terms just by having your mail server receive the message. Second, you
have violated those terms by having read the message and the
disclaimer. Such disclaimers are bad."

> emails, mine actually mean something. I think the shitter just got shat on?
> *Dances around playing a trumpet*
Yeah... a little im- and permature.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] I10N and toolserver subdomains

[Daniel Schwen]

> AFAIK most of us do so (including me), but we also don't want people
> to be stuck with a language of our automatic choice. So, the parameter

It is not automatic, you are not stuck,
you can configure it in your browser or operating system.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Daniel Schwen]

> hemlock# du -sh iip-cache-dschwen
> 201G    iip-cache-dschwen

Oh, right, I forgot about that (checked it yesterday and thought in
relation to the 3.7TB it was negligible...).
That directory contains multiresolution images generated from very
large images on commons for the zoomviewer [1].
Generating them takes up to more than a minute each, and many of them
actually had to be generated on an external server (as the process
uses a lot of memory for a short time and is frequently killed by
slayerd). Deleting them is not really an option (well it is, but it
would mean discontinuing the service).
Daniel

[1] http://toolserver.org/~dschwen/iip/wip.php?f=Seattle_7.jpg

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] user-store disk usage

[Daniel Schwen]

> dschwen          : 433.88G in 16212733 files

Uhm, I see 215G in /mnt/user-store/wikiminiatlas/, I thought that is
the lions share of my stuff. Where is the other half?
I'm running my tile cache prune script now. It fell out of my crontab
when I last moved my project to another server.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Golem issues

[Daniel Schwen]

On Wed, Mar 31, 2010 at 9:13 AM, Анастасия Львова  wrote:
> A major difference between isolated articles and the rest of articles
> with quality notes is that fixing the isolated requires editing other
> articles...
That is not an issue at all! You just extract the data from the edited
articles and use it to update your graph, which you can keep in
apermanent non-memory table.

On Wed, Mar 31, 2010 at 10:06 AM, Mike.lifeguard
 wrote:
> It is also quite a leap to call linking these articles an important kind
> of quality. It is obviously useful, but the Wikipedias will not die
Ok, I guess such posts by tooldevelopers here always have to be taken
with a grain of salt. Of course in my world _my_ tools are the most
important ones for Wikipedia too ;-).

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Golem issues

[Daniel Schwen]

Ok, I just had a really brief look at the output, but it strikes ma as
suboptimal to scan the entire database periodically rather than
performing incremental updates, by just pulling the subset of articles
that were edited since the last update and adjusting linkage
accordingly.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Golem issues

[Daniel Schwen]

> to obtain timely data. During these periods the number of isolated articles
> usually grows, and the growth gradually turns to decline once the Golem
> being started to work again. So, it means, that any idle period of Golem

Just out of curiosity. How exactly does Golem help here? It is quite a
leap from detecting isolated article clusters to actually linking
them. Can you please describe the process how Golem helps authors?

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

> is not possible due to a less free component (like a code library), the
> source code adopts the license of the library by default."

Adopting the license of the library by default makes not much sense. A
free license that allows linking unfree libraries would be the way to
go.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

@Eusebius:
>>> I would dare guessing that for most of
>>> the users it would make pretty much zero difference which free license
>>> (FSF approved) the choose for their projects.
> viral/nonviral could be a significant difference to some developers.
> Guillaume

I don't really see how even this is significant. It's not like any of
the tools on the TS, which are all pretty specialized niche programs,
would be interesting for derivative works or inclusion in commercial
software.


@Pathoschild:
> requests nor collects user credentials; users provide temporary access
> directly to me, knowing the risks of doing so. The Toolserver is
That looks like word play to me. You are proceccessing Wikimedia
account details on the toolserver (I'm assuming you use centralauth
token and session cookies? Just as bad.). That's all that matters to
me.

> The license indicates what freedoms the author intends for his tools.
> It makes no sense to license a tool for redistribution if you don't
> want it redistributed.
This whole discussion is not about redistribution, it is about
developers being able to pick up and rescue abandoned tools.

> Trust between Toolserver users is irrelevant
> here, since home directories are not world-readable.
If you read my initial proposal carefully (well actually just read it
at all), you'll see that I proposed just that.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

> to agree with having an opt-out function, mostly for legitimate
> concerns like Pathoschilds.

Pathoshilds concern is not really valid. 1) per Mike.lifeguards
comment, 2) we would not have to actively disseminate the code. The
issue is sharing between toolserver users. Potentially harmful tools
like pathoschild's (which seem to bee against TS rules anyways) cannot
be kept from being distributed just through license terms in any case.
Either you trust your fellow TS users, or if not, do not bring this
kind of software to the toolserver in the first place.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

> The default copyright stance, unless a licence specifies otherwise, is
> "All Rights Reserved". I don't think we have the right to enforce a
> licence that is all about freedom unless a user opts-in.
Of course we have. When a user obtains an account on the toolserver he
effectively enters contract with Wikimedia Deutschland. Of course
licensing can be tied to that. How can you suggest otherwise?!

> Closed source software can be as good as open source software - do
> remember that. And closed source software doesn't have to be
> commercial. While (imo) WM-DE should support free and open source
This is _completely_ besides the point. Please do not make this a
heise-forum-style idiology debate. There are purely pragmatic reasons
for free licensing here: 1) avoiding dying tools when maintainers
leave. 2) fostering synergies through code sharing

Of course you can take the stance to say "just write the tool again".
What a waste of time! Developer resources should be values higher than
this.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

> code, but whether or not to license code in such a way should remain a
> *free choice*.

Getting a TS account or not getting a TS account already is a free
choice. It is not unreasonable at all to tie certain policies to the
account. As a matter of fact this is already being done. [1]

[1] https://wiki.toolserver.org/view/Rules

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

> Maybe I didn't understand you properly, but wouldn't it be better to make a
> list of acceptable licenses, instead of a single license?

Those are minor details. And I would dare guessing that for most of
the users it would make pretty much zero difference which free license
(FSF approved) the choose for their projects.
P.S.: by making source world-readable I was referring to unix
permissions, which would of course only have an effect for people with
toolserver accounts.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Changes to expired accounts web hosting

[Daniel Schwen]

One solution would be to make free licensing a prerequisite to
obtaining a toolserver account.
And then make it mandatory to make all source-code world readable
(minus config files containing passwords of course).

> Currently we don't have a good policy for this.  Many (most?) tools have no
> copyright header or license, which makes it difficult to distribute them.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] Interwiki-Bots

[Daniel Schwen]

> Will a single instance of pywikipedia, even running on a fast box with
> a fat channel to pmtpa able to perform all edits needed? How many

There seems to be confusion about the meaning of "instance". As far as
I understand we are talking about a single codebase (possibly linked
to a multi user account on the toolserver). There is no limit on how
many processes can be run at the same time from that same code.

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] problem downloading (Templatetiger)

[Daniel Schwen]

Just out of curiosity, why are you using LIKE even though you do not
have any wildcards in your match string? Wouldn't = do the same? Or is
MySQL optimizing so that it doesn't make a difference?

> (SELECT `Value` FROM `pub_tt1_de` WHERE `entry_name` LIKE 'ALTERNATIVNAMEN' 
> AND `tp_name` LIKE 'Personendaten' AND `name` = a LIMIT 1   ) ALTERNATIVNAMEN,

___
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org)
https://lists.wikimedia.org/mailman/listinfo/toolserver-l
Posting guidelines for this list: 
https://wiki.toolserver.org/view/Mailing_list_etiquette

Re: [Toolserver-l] The Norwegian toolserver

[Daniel Schwen]

> receives come with strings attached, and Wikimedia PL can only spend
> that money _within Poland_. None of their doing, it's the law, you
> can't do anything about it. It goes with their non-profit status. In

I find this hard to believe. By the same logic it would be impossible
for a polish non-profit to perform humanitarian aid in development
countries.

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] Next level of stabilization

[Daniel Schwen]

> the Verein separately spent ~20'000 EUR on a server to replicate
> Wikimedia uploads in Amsterdam.

Oh! Is that server accessible from the toolserver cluster? It would make a few 
applications quite a bit easier/faster (I have a bot that detects animated 
GIFs, and one that extracts GPS data from the EXIF block in files (no, its 
not in the DB!))

Dschwen

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] WikiMiniAtlas (was: Next level...)

[Daniel Schwen]

> and I doubt this is the best way for WikiMiniAtlas, since there
> are so many different ways that the coord template can be called.
> Just consider all the lat/lat_min parameters to infobox templates.
> Shouldn't you just dig out from the external links table, all the
> links to the stable.toolserver.org/geohack/ and parse the
> coordinates from those URLs?
Yes I should, and I thought I already told you that this is what I'm doing for 
the english wikipedia and for commons. And I also set up a page explaining 
why this is quiite a lot of work, and how people can help me with this, if 
they want faster updates for their language. 
http://meta.wikimedia.org/wiki/WikiMiniAtlas/CoordinateProcessing
I guess I'll have to make this more public


> And when WikiMiniAtlas is invoked from an article, such as
> Grängesberg, the call to the toolserver (that fetches the map
> tiles) could perhaps be used for updating the coordinate for that
> article.  We can be pretty sure that those who update a coordinate
> will pop up the WikiMiniAtlas to see they made no mistake.
Yeah, that's not a bad idea. I even used to have a Javascript gadget that sent 
the data to the toolserver, whenever a coordinate in an article was updated. 
In any case, the red dot, is generated on the the client side and will always 
use the most up to date coordinates.

Dschwen

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] Betawiki translations - cool, but how?

[Daniel Schwen]

Ok, sounds interesting. And I keep hearing about Betawiki, found it through 
Google, but I don't see any manual for getting a project onto Betawiki.
I'd be very interested in moving the WikiMiniAtlas translations there.

-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

[Toolserver-l] potential abuse by 84.114.164.84

[Daniel Schwen]

While looking at the access statistics of the Query-to-map tool I discovered 
that a large portion of the hits are generated by one single austrian DSL IP 
(9 hits yesterday (9% of all hits that day), 55000 hits so far today, ).

Seemingly random queries to lots of tools. Example:

84.114.164.84 - - [11/Jan/2009:14:03:43 
+] "HEAD 
/%7Ekolossos/wp-world/umkreis.php?la=pt&lon=16.285&lat=48.13&rang=50&map=1 
HTTP/1.1" 200 0 "-" "Jakarta Commons-HttpClient/3.1"
84.114.164.84 - - [11/Jan/2009:14:03:49 
+] "HEAD /~para/earth.php?latdegdec=48.13&londegdec=16.285&scale=30 
HTTP/1.1" 301 0 "-" "Jakarta Commons-HttpClient/3.1"
84.114.164.84 - - [11/Jan/2009:14:03:49 
+] "HEAD 
/~kolossos/wp-world/umkreis.php?la=nl&lon=16.285&lat=48.13&rang=50&map=1 
HTTP/1.1" 301 0 "-" "Jakarta Commons-HttpClient/3.1"
84.114.164.84 - - [11/Jan/2009:14:03:50 
+] "HEAD 
/~kolossos/wp-world/umkreis.php?la=pt&lon=16.285&lat=48.13&rang=50&map=1 
HTTP/1.1" 301 0 "-" "Jakarta Commons-HttpClient/3.1"

What is going on here? Proxy?
Daniel

P.S.: there is a webserver on that IP. Looks like a broken Mediawiki 
installation, a broken forum installation. No further info though.

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] mysqldump

[Daniel Schwen]

> > I need to make a query between itwiki_p (sql-s2) and enwiki_p
> > (sql-s1). Is it possible? I think I need to download an ufficial dump

> Why can't you just run the query across databases?  What query are you
> trying to run?

Cross-database queries I know, but cross-server queries are new to me. Is that 
even possible?

-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] no more thumbnails (access denied to thumb.php)

[Daniel Schwen]

> > And where can I find the source code? It seems your redirect tool does
> > not work when supplying HTTP Range headers.
>
> Apparently, in SVN:
> https://fisheye.toolserver.org/browse/dab/TS-thumb/

Ok, pretty trivial. In case someone ever needs a python snippet for let's say 
a pywikipedia based bot:

import hashlib
import urllib2
m = hashlib.md5()
m.update( name )
h = m.hexdigest()
url = "http://upload.wikimedia.org/wikipedia/commons/thumb/%s/%s/%s/120px-%s"\
% ( h[0], h[0:2], urllib2.quote(name), urllib2.quote(name) )

Just take "name" straight as mysql returns it from a db query, otherwise the 
hash will most likely be wrong.
-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] no more thumbnails (access denied to thumb.php)

[Daniel Schwen]

> I'm done for first. You can find my programm at
> http://toolserver.org/tsthumb/tsthumb

And where can I find the source code? It seems your redirect tool does not 
work when supplying HTTP Range headers. 
Daniel
-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] no more thumbnails (access denied to thumb.php)

[Daniel Schwen]

> What about sending to the full sizeimage if a size bigger than the image
> is requested?
> Also, as you're issuing a redirect, what are the risks if someone was
> able to upload a malicious file to the servers? (the thumbnailing
> process may guard against it)

Make it only deliver the original if explicitly asked for it. Then it would 
also be a replacement for Special:Filepath

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] new login server

[Daniel Schwen]

> this server is for everything except web serving.  all your bots, tools,
> etc. must now be run here.
[..]
> /home is shared between both servers, so there is no need to copy any files
> over.  http://toolserver.org/ URLs still point to hemlock.
[..]]
> after 1 week from today (June 19), we will start killing all tools which
> run on hemlock.

So people who use user-crontabs must check the hostname now?

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] Windows toolserver

[Daniel Schwen]

> >  ... Shelling out even
> > one dollar just because someone is to lazy (sorry) to learn new things
> > seems unacceptable to me.
>
> What makes you rightful to say in other words that Windows programmers are
> lazy slackers? Who are you?

Oh geez! AGF, for crying out loud! Neither was this a personal attack, nor was 
I playing that annoying old Linux vs. Windows record (even if you want to 
make me sound like it).

Read my post again!

I was asking for examples where a valuable tool developed on windows cannot 
simply be run on Linux. You didn't provide any. 

Now Sebmol seems to have such an example, which is perfectly fine. 
Neither am I opposed to the general idea of having a Windows toolserver, nor 
do I deem Windows "evil".

The gist is: if conversion is possible no money should be spent. If it is not 
(with reasonable effort) and the tools provide substantial value, then fine 
by me.
-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] Windows toolserver

[Daniel Schwen]

> It's the question of reusability of software. I know about people
> (including myself) who wrote stuff running on Windows. And they did not
> write it on Linux just because either they didn't need it or simply because
> they don't understand Linux programming that much or even not at all. Now,
> does that mean we should throw away all the stuff which has been done? It's
> unlikely to happen those users will port it on Linux. So the question  is,
Then ask for help!

What is there that can only be done on Windows but not on Linux? No examples 
have been presented yet. 

If it just boils down to "I feel cozy using C# on windows and never had a look 
at mono", then I'm sorry, that doesn't cut it for me.
I had to acquire a considerable amount of expertise and put in a substantial 
amount of time to do the stuff I'm doing on the toolserver. Shelling out even 
one dollar just because someone is to lazy (sorry) to learn new things seems 
unacceptable to me.

-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] protip: i/o and infinite loops in php

[Daniel Schwen]

> > That would be the obvious choice. But unfortunately there seems to be no
> > way to realize that in PHP. And furthermore if the fifo has no listener
> http://nl2.php.net/manual/fi/function.stream-select.php ?

How do I get a stream object to pass to stream_select? fopen? fopen is already 
blocking if my fifo listener is dead.
-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] protip: i/o and infinite loops in php

[Daniel Schwen]

> > aborting the php script if the write would be blocking.
> Maybe using non-blocking I/O would help too ?:)

That would be the obvious choice. But unfortunately there seems to be no way 
to realize that in PHP. And furthermore if the fifo has no listener there is 
no point in writing the data. Having the data queued until the listener comes 
up again would just create a huge backlog of work (map tiles to be rendered) 
which no one would care about (since the requests for those tiles timed out 
anyways).
-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/toolserver-l

Re: [Toolserver-l] protip: i/o and infinite loops in php

[Daniel Schwen]

> why not: because if the i/o fails, your script will run forever, using
> 100% cpu, doing nothing.
Mh, yeah, I had a similar problem with my scripts. No infinite loop,  but a 
blocking write to a fifo, as my fifo listener process was killed off. 
I wasted lots of memory with my stale php processes. Sorry about that!
I fixed it by adding checks for the fifo listener (using pid files) and 
aborting the php script if the write would be blocking.
-- 
[[en:User:Dschwen]]
[[de:Benutzer:Dschwen]]
[[commons:User:Dschwen]]

___
Toolserver-l mailing list
Toolserver-l@lists.wikimedia.org
http://lists.wikimedia.org/mailman/listinfo/toolserver-l