Re: [Wikitech-l] Google Summer of Code 2009

2009-03-11 Thread Nicolas Dumazet
Yeah! I was part of the mixed luck from last year, and honestly, I get warm feelings when reading the friendly http://www.mediawiki.org/wiki/Summer_of_Code_2009/Application_template ;) I don't know if anyone here is willing to be a mentor this year, but please go ahead, and try to help a

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Aryeh Gregor: I don't think the toolserver is used for backups. it is, but only in the sense that it's our only off-site copy of the database. it was not created to act as a backup... At least I hope it's not, given its reliability (which is

Re: [Wikitech-l] how to convert the latin1 SQL dump back into UTF-8?

2009-03-11 Thread Tei
note to self: look into the code that order text (collation) in mediawiki, has to be fun one :-) -- -- ℱin del ℳensaje. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Rohde: In particular, I think it is useful to separate tools from analysis. why? Tools need high availability and low lag relative to the live site, but analysis doesn't care if it gets out of date and should use scheduling etc. to

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Andrea Forte: Let me know if you have a grant proposal you'd like help with! well, i'm still not sure what exactly people need. perhaps the various academic people could produce a list of what they want to do on the toolserver and what's missing at

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Brian
I vote for making the toolserver the head-node to a much larger beowulf cluster that has a well configured job scheduler. The data that needs to be crunched is already right there - it makes sense to put a research cluster there as well. There will always be a limited supply of resources. Perhaps

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian: I vote for making the toolserver the head-node to a much larger beowulf cluster that has a well configured job scheduler. so the issue is that more CPU is needed to run the research jobs? how much more? do you have an example of a job and

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Brian
Sure - creating a lucene index of the entire revision history of all wikipedia's for a WikiBlame extension. More realistically (although I would like to do the above) a natural language parse of the current revision of the english wikipedia. Based on the supposed availability of this hardware,

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread River Tarnell
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Rohde: The starting point is providing full-text history availability and once you have that there are a number of different projects (like wikiblame) which would desire to pull and process every revision in some way. okay, so full text

Re: [Wikitech-l] how to convert the latin1 SQL dump back into UTF-8?

2009-03-11 Thread Daniel Kinzler
Tei schrieb: note to self: look into the code that order text (collation) in mediawiki, has to be fun one :-) There is none. Sorting is done by the database. That is to say, in the default comnpatibility mode, binary collation is used - that is, byte-by-byte comparison of UTF-8 encoded data.

Re: [Wikitech-l] how to convert the latin1 SQL dump back into UTF-8?

2009-03-11 Thread Aryeh Gregor
On Wed, Mar 11, 2009 at 6:14 AM, Daniel Kinzler dan...@brightbyte.de wrote: There is none. Sorting is done by the database. That is to say, in the default comnpatibility mode, binary collation is used - that is, byte-by-byte comparison of UTF-8 encoded data. Which sucks. But we are stuck with

Re: [Wikitech-l] Google Summer of Code 2009

2009-03-11 Thread Brion Vibber
On 3/11/09 4:55 AM, Chad wrote: Maybe someone can finish HTMLDiff from last summer. That would be awesome. :) The inline diff view has pluses and minuses, but it's a very nice tool for the toolbox... (Sometimes you really want to see the source in place, but sometimes it's wa easier to

Re: [Wikitech-l] Google Summer of Code 2009

2009-03-11 Thread Brion Vibber
On 3/10/09 10:59 PM, Nicolas Dumazet wrote: I don't know how to put it nicely, but the key for GSoC to succeed, on the mentor/senior devs side, is just to be *very* available. Easy thing to say, I know, but that would be nice to keep this in mind if we plan to host students this year. It's

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Aryeh Gregor
On Wed, Mar 11, 2009 at 11:20 AM, Brion Vibber br...@wikimedia.org wrote: Quite so. :) Replication is fantastic against outright failure, but by itself doesnt help agaibst daya loss within the system which gets replicated right alobg with it we're working on ensuring we've got regular

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Brion Vibber
On 3/11/09 9:43 AM, Aryeh Gregor wrote: On Wed, Mar 11, 2009 at 12:35 PM, Platonidesplatoni...@gmail.com wrote: I know. That's precisely what i'm addressing. From your email, WMF is reorganising their databases so the toolserver can get more admins (less private data is replicated/stored at

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Platonides
River Tarnell wrote: it seems more useful to provide the text in uncompressed form, instead of the MediaWiki internal form that's almost impossible to work with. does that seem reasonable? The tools should get the text in uncompressed form. The interface to do that is not so important. Given

Re: [Wikitech-l] Fwd: Re: [Foundation-l] Report to the Board of Trustees: December 2008

2009-03-11 Thread Thomas Dalton
2009/3/11 Brion Vibber br...@wikimedia.org: On 3/11/09 8:21 AM, Chad wrote: On Wed, Mar 11, 2009 at 9:25 AM, Lars Aronssonl...@aronsson.se  wrote: Sue Gardner wrote: Report to the Wikimedia Foundation Board of Trustees [...]  From December 9-15, Jimmy Wales and Sue Gardner visited India. It

Re: [Wikitech-l] It makes more overload to delete system massages in local wiki's MediaWiki namespace pages?

2009-03-11 Thread Roan Kattouw
mizusumashi schreef: Hello everyone. I'm a sysop of Japanese Wikipedia. I'm going to delete many MediaWiki namespace pages that have same contents as betawiki. I think this work will make less operation cost. I hava a question about caching and sever's overload. My colleague said that

Re: [Wikitech-l] It makes more overload to delete system massages in local wiki's MediaWiki namespace pages?

2009-03-11 Thread Platonides
Don't worry about performance, delete from the wiki and use betawiki. http://en.wikipedia.org/wiki/Wikipedia:Don%27t_worry_about_performance The files get heavily cached. A page-existence check is faster than retrieving tha page, but in this case, all mediawiki messages are cached at memcached.

Re: [Wikitech-l] It makes more overload to delete system massages in local wiki's MediaWiki namespace pages?

2009-03-11 Thread Gerard Meijssen
Hoi, The advantage of deleting the messages that are the same in translatewiki.net (formerly Betawiki) and the local messages is that when a message CHANGES, a change made at translatewiki.net will have an effect on your local system as well. It is for this reason that it is a good idea to have as

Re: [Wikitech-l] Fwd: Re: [Foundation-l] Report to the Board of Trustees: December 2008

2009-03-11 Thread Brion Vibber
On 3/11/09 9:59 AM, Thomas Dalton wrote: 2009/3/11 Brion Vibberbr...@wikimedia.org: Is there a fix for this bug? It was fixed several years ago, shouldn't happen on anything current. :P March 2009 seems pretty current to me... Hence shouldn't and :P -- brion

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Tim Starling
K. Peachey wrote: Please report urgent system administration issues to IRC, specifically #wikimedia-tech on irc.freenode.net. -- Tim Starling Not everyone knows how to use IRC, i would recommend that you recommend people report to bugzilla with urgent tags instead. That would just be a

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Chad
On Wed, Mar 11, 2009 at 5:59 PM, Tim Starling tstarl...@wikimedia.org wrote: K. Peachey wrote: Please report urgent system administration issues to IRC, specifically #wikimedia-tech on irc.freenode.net. -- Tim Starling Not everyone knows how to use IRC, i would recommend that you recommend

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Chad
On Wed, Mar 11, 2009 at 6:16 PM, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi, If this is true, you just provided the best argument why the setup of Bugzilla should be changed. You do not want to have the tools that report on the availability of your production servers to be dependent

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Gerard Meijssen
Hoi, People _do_ post problems using Bugzilla.. What you are saying is that the _intention_ is not to use it for urgent sysadmin stuff. New bugzilla bugs _are_ posted on IRC in order to get people's attention. This real time attention negates your assertion that it is not treated as urgent. grin

Re: [Wikitech-l] It makes more overload to delete system massages in local wiki's MediaWiki namespace pages?

2009-03-11 Thread Chad
On Wed, Mar 11, 2009 at 6:36 PM, Aryeh Gregor simetrical+wikil...@gmail.com wrote: On Wed, Mar 11, 2009 at 12:57 PM, mizusumashi mizusuma...@coda.ocn.ne.jp wrote: I hava a question about caching and sever's overload.  My colleague said that local wiki's MediaWiki namescape pages may make less

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Aude
On Tue, Mar 10, 2009 at 9:54 AM, Tim Starling tstarl...@wikimedia.orgwrote: Please report urgent system administration issues to IRC, specifically #wikimedia-tech on irc.freenode.net. -- Tim Starling Certainly IRC is the best way to inform developers and sys admins of a problem with

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Chad
On Wed, Mar 11, 2009 at 6:38 PM, Gerard Meijssen gerard.meijs...@gmail.com wrote: Hoi, People _do_ post problems using Bugzilla.. What you are saying is that the _intention_ is not to use it for urgent sysadmin stuff. New bugzilla bugs _are_ posted on IRC in order to get people's attention.

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread DaB.
Am Tuesday 10 March 2009 01:07:36 schrieb phoebe ayers: their Wikipedia-related research has been put on hold for a few months because of the delay. (It seems like there is a big backlog of account requests right now and only one person working on them?) hello, I'm DaB. and I'm the lazzy guy

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Brion Vibber
On 3/11/09 3:41 PM, Tim Starling wrote: Gerard Meijssen wrote: 2009/3/11 Chadinnocentkil...@gmail.com wrote: Not to mention, depending on the error it could be entirely possible that Bugzilla is down as well. If this is true, you just provided the best argument why the setup of Bugzilla

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Morten Warncke-Wang
Hello everyone. We started the conversation with Phoebe about the possibility of a research-oriented toolserver that could be used by researchers who wish to explore novel gadgets or other tools for Wikipedia users. The toolserver could provide back-end support for these gadgets. By the phrase

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Gerard Meijssen
Hoi, Brion thank you. Gerard 2009/3/12 Brion Vibber br...@wikimedia.org On 3/11/09 3:41 PM, Tim Starling wrote: Gerard Meijssen wrote: 2009/3/11 Chadinnocentkil...@gmail.com wrote: Not to mention, depending on the error it could be entirely possible that Bugzilla is down as well. If

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread John Doe
The current toolserver user base is always willing to help. I for one am willing to review and run queries on the database if requested. I also am a very active python programmer and can use that to assist. If you have requests let me know. (Unless Im losing it) there have been accounts that are

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Aryeh Gregor
On Wed, Mar 11, 2009 at 7:05 PM, Morten Warncke-Wang mor...@cs.umn.edu wrote: The major difference from our perspective is how applications for new accounts would be handled.  Our idea is to be able to hand out accounts based around the likelihood of effective research, rather than on

Re: [Wikitech-l] research-oriented toolserver?

2009-03-11 Thread Daniel Kinzler
The major difference from our perspective is how applications for new accounts would be handled. Our idea is to be able to hand out accounts based around the likelihood of effective research, rather than on visibility within Wikipedia, or on the usefulness of the resulting tool to the larger

Re: [Wikitech-l] Illegal mix of collations after upgrade to 1.14

2009-03-11 Thread Aran
Hi I've tried upgrading my 1.11 to 1.14 and get this illegal mix of collations error. I went through the normal upgrade procedure first but this failed, so I then tried exporting as XML and importing into a completely fresh 1.14 install, and still I get the error! I've found that by setting

Re: [Wikitech-l] Wikipedia is full

2009-03-11 Thread Thomas Dalton
2009/3/11 Aude audeviv...@gmail.com: On Tue, Mar 10, 2009 at 9:54 AM, Tim Starling tstarl...@wikimedia.orgwrote: Please report urgent system administration issues to IRC, specifically #wikimedia-tech on irc.freenode.net. -- Tim Starling Certainly IRC is the best way to inform

Re: [Wikitech-l] Fwd: Re: [Foundation-l] Report to the Board of Trustees: December 2008

2009-03-11 Thread Thomas Dalton
2009/3/11 Brion Vibber br...@wikimedia.org: On 3/11/09 9:59 AM, Thomas Dalton wrote: 2009/3/11 Brion Vibberbr...@wikimedia.org: Is there a fix for this bug? It was fixed several years ago, shouldn't happen on anything current. :P March 2009 seems pretty current to me... Hence shouldn't and

Re: [Wikitech-l] Fwd: Re: [Foundation-l] Report to the Board of Trustees: December 2008

2009-03-11 Thread Casey Brown
On Wed, Mar 11, 2009 at 8:37 PM, Thomas Dalton thomas.dal...@gmail.com wrote: Hence I'll fix it, give me a minute!? (Or, more likely, I'll tell someone to fix it, give them a minute!) Comments from developers about bugs really should be about fixing them... Saying it shouldn't be happening