Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
On Tue, Mar 31, 2009 at 9:57 AM, howard chen wrote: > Given that the current dump process is having problem, why not provide > a simple fix such as providing raw table format , SQL files or even > CSV files? Please pardon this newbie question: is there a succinct explanation of what the problem is with the current Wikipedia dump process? //Ed ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] add media wizard in testing / update
The add media Wizard is in testing see blog post: http://metavid.org/blog/2009/03/27/add-media-wizard-and-firefogg-on-test-wikimediaorg/ If no one objects (or has any blocker bugs that I have missed) I will add the gadget option for firefogg / add media wizard to commons shortly. (for wider testing) We could also add the basic commons search as a gadget for wikipedia proper. (making commons image inserts much easier) more about add_media_wizard: http://www.mediawiki.org/wiki/Extension:Add_Media_Wizard The searching remote repository and in-line upload and template via url component that is working on test.wikimedia.org is not as easy to push out. The implementation on test.wikimedia.org is limited to files that it can copy in less than 30 seconds (php time out ) and is parsing html page output to determine errors. To move the add media wizard searching and insert into production we will need to deploy the new-upload branch. I will be targeting fixes to upload bugs 18200, 18201, 18202 to the new-upload branch. Looking beyond English language gadgets ... Localization is dependent on the javascript script server which I will create a patch for core shortly. more about js server: http://www.mediawiki.org/wiki/Extension:ScriptLoader peace, --michael ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] which test wiki is current with SVN?
Roan Kattouw schrieb: > > TranslateWiki.net usually runs a pretty recent version. Right now > they're running r49060, which is less than 9 hours old. AFAIK, > Siebrand is the one who scaps TW, and he's in CEST (Amsterdam/Paris > time), so scaps may happen at weird times depending on your timezone. > I scaps TW too :) But my scap times depends on my online times and I am living in the same timezone as Siebrand. Raymond. signature.asc Description: OpenPGP digital signature ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] which test wiki is current with SVN?
The thing is, you should be sure that your commit doesn't break anything, so this shouldn't be too big a problem. -X! On Mar 31, 2009, at 10:26 AM [Mar 31, 2009 ], jida...@jidanni.org wrote: > What test wiki other than one's own runs the current SVN version? > Even test.wikipedia.org is "only" at r48811. > How does one confirm a problem is not just local before posting a bug > without waiting a few days for test.wikipedia.org to catch up? > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A proposal to simplify and improve footnote markup in Wikipedia
On Tue, Mar 31, 2009 at 12:02 PM, Håkon Wium Lie wrote: > It's easy enough to rewrite, and the worst-case scenario is a minor > stylistic change for some users. In the best case, the markup and the > style sheet are changed in sync. In theory. The problem is organizational: the people who change the software, the people who change the stylesheets on any particular wiki, and the people who sync new versions of the software to the site constitute a few hundred different groups, mostly pairwise disjoint. There are over a thousand Wikimedia wikis last I heard, and to be sure we didn't break anything we'd have to review all of their stylesheets and fix them if necessary. It would be interesting if we had a procedure for this, however. It should suffice to write a script to grep all the stylesheets on all sites for appropriate rules, and ask a steward or other global sysop to add the replacement rules before the software is synced (in addition to the old ones, so there's no period where neither set works). This is more convenient now than it used to be, since we do have a unified login and global sysops. Now, why don't we allow in wikimarkup? . . . ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
On 3/31/09 8:21 AM, Domas Mituzas wrote: >> (as you know XML generation and parsing really >> take time...) >> > > > I didn't know that. Ever tried SAX? > > Indeed, not all XML software is slow... http://dotnot.org/blog/archives/2008/02/ - Trevor ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A proposal to simplify and improve footnote markup in Wikipedia
Also sprach Aryeh Gregor:
> > http://www.princexml.com/howcome/2009/wikipedia/ref/
> This looks good overall.
Thanks.
> 1) A problem we often tend to have with this kind of thing is that
> users write lots of custom CSS and JS that depends on the exact
> elements used, which breaks if we change the markup. For instance, I
> find the following rules in the English Wikipedia's custom stylesheet:
>
> sup.reference:target {
> background-color: #DEF;
> }
> /* stop references with groups from line-breaking */
> sup.reference a {white-space: nowrap;}
The selectors would have to be rewritten to match the revised markup:
a.ref:target { background-color: #DEF }
a.ref { white-space: nowrap }
>
>
> These would break if we got rid of the , on the English Wikipedia
> and possibly on tons of other wikis as well. This makes me reluctant
> to do that.
It's easy enough to rewrite, and the worst-case scenario is a minor
stylistic change for some users. In the best case, the markup and the
style sheet are changed in sync.
> 2) I notice you used "font-size: 85%; vertical-align: 35%" instead of
> the CSS2.1 recommended value for sup, "font-size: 83%; vertical-align:
> sup". Is there any particular reason for this?
Indeed, CSS 2.1 proposes [1]:
sup { vertical-align: super }
However, with this value, some legacy browsers (IE6/IE7) place the
footnote call slightly higher than Wikipedia currently uses. So, the
35% value was chosen as it seems to result in "pixel-perfect"
renditions. I would be equally happy with the "super" value.
The 85% value is semi-arbitrary, I've changed it to 83% which seems to
work equally well.
[1] http://www.w3.org/TR/CSS21/sample.html
> Specifically, one important objective for such a change would be
> that line-spacing shouldn't be changed, and I'd have to test that
> in various browsers to avoid regression there.
Yes.
> 3) Some of the changes you recommend actually aren't software issues,
> but customizations by the particular wiki's sysops. In particular,
> the extra s around the square brackets are specific to enwiki:
>
> http://en.wikipedia.org/w/index.php?title=MediaWiki:Cite_reference_link&action=history
>
> So is the :
>
> http://en.wikipedia.org/w/index.php?title=MediaWiki:Cite_references_link_one&action=history
Interesting. No good reasons are given. Hmm.
> In the default software, they don't exist. To get rid of them, you'd
> want to make a suggestion on wikien-l, the enwiki Village Pump, or a
> similar forum. wikitech-l is mostly for the developers and sysadmins,
> who can't really affect what particular communities choose to do in
> this sort of thing.
Thanks for pointing me in the right direction.
> 4) The reason we have an inside a turns out to be because in
> wikimarkup, you can't give custom attributes to . The markup used
> for the message is wikimarkup, which simply cannot achieve your
> suggested HTML output. We'd have to move this to raw HTML to allow
> it, which would mean hardcoding it into the software, since we don't
> like to give sysops access to output raw HTML (to reduce the chance of
> invalid markup -- they can add arbitrary JS anyway, so XSS isn't
> really an issue). This would probably annoy people, since they've
> already customized it. Or we'd have to add support for explicit
> to wikimarkup -- maybe a good idea, but also a bigger project.
I'm happy to be put in the "bigger project" category -- I'm here for
the long term -- centuries :-)
> I do like your proposed markup, and think it would be great if we
> could move to simpler and more semantic markup overall, but it's not
> clear to me how to best go about implementing most of it. I've at
> least removed the empty title="" attribute, in r49075. Of the rest,
> some I can't do: it's up to enwiki sysops. And one bit (giving
> attributes to the element) wouldn't work at all without some
> potentially significant unrelated changes. But it would be nice if we
> could move more in this direction, definitely.
Thanks for your considered reply.
Cheers,
-h&kon
Håkon Wium Lie CTO °þe®ª
howc...@opera.com http://people.opera.com/howcome
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] PGP-keysign at the tech/chapter-meeting
Hello all, I think that when such a number of people come together it would be nice to have a key-signing in Berlin. If you have no idea, what a key-signing is, look at the wikipedia-article [[en:Key_signing_party]]. If you don't own a pgp-key yet and are an linux-user there are several how-tos on the net to get one fast (there are how-tos for windows-users too, but it's more complex, but that doesn't need to stop you). Because there is no time (and place) for a hash-methode-keysigning (you know, all standing in a line for hours ;)), I would organise a list-methode-keysigning. That means that you send me *Your nick (if you have one) *Your realname (optional, but some people don't sign non-realname-keys) *Your keynumber *Your key-hash *Your key (if it is not on normal key-servers) to p...@daniel.baur4.info. I will make a list of that data. Then I (respectively the verein) will print out several copies of that list and place them at central places (like the reception or the c-base) for hand-out. You can also place sticker on your nameplate (to let people easier find you) if you like (I have colorful sticker-dots here ;)). You will take such an list and start looking for other people, that stand on the list, check your hash on his/her list and his/her hash on your list and mark your name as checked on his/her list and his/her name on your list if all is right. Then continue until all entries on your list are checked (or the meeting is over). It would be very nice if many people would participate to increase the level of trust (at least the key-trust) between us. Follow up to foundation-list. Sincerly, DaB. P.S: Even if you come just to the party on saturday, you can take part! 2.P.S: I organize a key-signing for the first time, so please be patient if I did anything wrong. -- Diese eMail sollte mit dem PGP-Schlüssel 0x2D3EE2D42B255885 digital signiert sein. Bitte beachten Sie, das unsignierte eMails beliebig gefälscht sein können. Achten Sie daher auf Signaturen. signature.asc Description: This is a digitally signed message part. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
> (as you know XML generation and parsing really > take time...) I didn't know that. Ever tried SAX? -- Domas Mituzas -- http://dammit.lt/ -- [[user:midom]] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] which test wiki is current with SVN?
Hi! > What test wiki other than one's own runs the current SVN version? mediawiki-l@ ? > Even test.wikipedia.org is "only" at r48811. test.wikipedia is staging ground, not mediawiki development testbed. > How does one confirm a problem is not just local before posting a bug > without waiting a few days for test.wikipedia.org to catch up? by employing such things as analysis and common sense. -- Domas Mituzas -- http://dammit.lt/ -- [[user:midom]] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] which test wiki is current with SVN?
2009/3/31 : > What test wiki other than one's own runs the current SVN version? > Even test.wikipedia.org is "only" at r48811. > How does one confirm a problem is not just local before posting a bug > without waiting a few days for test.wikipedia.org to catch up? TranslateWiki.net usually runs a pretty recent version. Right now they're running r49060, which is less than 9 hours old. AFAIK, Siebrand is the one who scaps TW, and he's in CEST (Amsterdam/Paris time), so scaps may happen at weird times depending on your timezone. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A proposal to simplify and improve footnote markup in Wikipedia
On Tue, Mar 31, 2009 at 5:52 AM, Håkon Wium Lie wrote:
> In the quest to simplify and improve Wikipedia's HTML code, the turn
> has come to footnotes. Here is a proposal that describes how the
> number of elements needed to represent footnotes can be halved:
>
> http://www.princexml.com/howcome/2009/wikipedia/ref/
>
> As far as I can tell, the proposed markup works in legacy browsers,
> too.
This looks good overall. Some comments:
1) A problem we often tend to have with this kind of thing is that
users write lots of custom CSS and JS that depends on the exact
elements used, which breaks if we change the markup. For instance, I
find the following rules in the English Wikipedia's custom stylesheet:
sup.reference:target {
background-color: #DEF;
}
/* stop references with groups from line-breaking */
sup.reference a {white-space: nowrap;}
These would break if we got rid of the , on the English Wikipedia
and possibly on tons of other wikis as well. This makes me reluctant
to do that.
2) I notice you used "font-size: 85%; vertical-align: 35%" instead of
the CSS2.1 recommended value for sup, "font-size: 83%; vertical-align:
sup". Is there any particular reason for this? Specifically, one
important objective for such a change would be that line-spacing
shouldn't be changed, and I'd have to test that in various browsers to
avoid regression there.
3) Some of the changes you recommend actually aren't software issues,
but customizations by the particular wiki's sysops. In particular,
the extra s around the square brackets are specific to enwiki:
http://en.wikipedia.org/w/index.php?title=MediaWiki:Cite_reference_link&action=history
So is the :
http://en.wikipedia.org/w/index.php?title=MediaWiki:Cite_references_link_one&action=history
In the default software, they don't exist. To get rid of them, you'd
want to make a suggestion on wikien-l, the enwiki Village Pump, or a
similar forum. wikitech-l is mostly for the developers and sysadmins,
who can't really affect what particular communities choose to do in
this sort of thing.
4) The reason we have an inside a turns out to be because in
wikimarkup, you can't give custom attributes to . The markup used
for the message is wikimarkup, which simply cannot achieve your
suggested HTML output. We'd have to move this to raw HTML to allow
it, which would mean hardcoding it into the software, since we don't
like to give sysops access to output raw HTML (to reduce the chance of
invalid markup -- they can add arbitrary JS anyway, so XSS isn't
really an issue). This would probably annoy people, since they've
already customized it. Or we'd have to add support for explicit
to wikimarkup -- maybe a good idea, but also a bigger project.
I do like your proposed markup, and think it would be great if we
could move to simpler and more semantic markup overall, but it's not
clear to me how to best go about implementing most of it. I've at
least removed the empty title="" attribute, in r49075. Of the rest,
some I can't do: it's up to enwiki sysops. And one bit (giving
attributes to the element) wouldn't work at all without some
potentially significant unrelated changes. But it would be nice if we
could move more in this direction, definitely.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A proposal to simplify and improve footnote markup in Wikipedia
Also sprach Nikola Smolenski: > > http://www.princexml.com/howcome/2009/wikipedia/ref/ > > > > As far as I can tell, the proposed markup works in legacy browsers, > > too. > > I like it, except adding square brackets with CSS. Uses that are the > most likely not to support CSS are exactly the uses where the brackets > are most needed. Good. I think it's ok to keep the brackets in the HTML code -- it's a common style and we can affort the extra two characters. This is also what the study proposes. Cheers, -h&kon Håkon Wium Lie CTO °þe®ª howc...@opera.com http://people.opera.com/howcome ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] which test wiki is current with SVN?
What test wiki other than one's own runs the current SVN version? Even test.wikipedia.org is "only" at r48811. How does one confirm a problem is not just local before posting a bug without waiting a few days for test.wikipedia.org to catch up? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
howard chen schrieb: > Hello, > > Given that the current dump process is having problem, why not provide > a simple fix such as providing raw table format , SQL files or even > CSV files? Ooooh! You meant for Wikipedia, not for individual MediaWiki installations? Well that makes a lot more sense. We call DumpHTML our dump process and it isn't running smoothly for us ATM and clients want flattened HTML from the wikis to take around to trade shows and whatever. Sorry for being confused! -Courtney ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
On Tue, Mar 31, 2009 at 10:02 AM, Christensen, Courtney wrote: > -Original Message- > Given that the current dump process is having problem, why not provide > a simple fix such as providing raw table format , SQL files or even > CSV files? > Howard, > > Can't you get the SQL files from running mysqldump from the command line? > Why does something new need to be created? I hope I'm not being dense, but I > don't understand what new niche you are asking to fill. Because the data (text) isn't in a single database, even for a single project, it is spread across a large number of machines. It's also in a mixture of bizarre internal formats. The file format it pretty much irrelevant to the 'cost' of producing a dump. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
-Original Message- Given that the current dump process is having problem, why not provide a simple fix such as providing raw table format , SQL files or even CSV files? Howard ___ Howard, Can't you get the SQL files from running mysqldump from the command line? Why does something new need to be created? I hope I'm not being dense, but I don't understand what new niche you are asking to fill. Thanks! -Courtney ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
howard chen schrieb: > Hello, > > Given that the current dump process is having problem, why not provide > a simple fix such as providing raw table format , SQL files or even > CSV files? Because that would contain private data. It needs to be filtered first. And for that, the text blobs have to be uncompressed and pulled apart first, so individual revisions can be handeled. When all that is done, it can just as well be written as XML. XML as such is not the problem. -- daniel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Providing simpler dump format (raw, SQL or CSV)?
Hello, Given that the current dump process is having problem, why not provide a simple fix such as providing raw table format , SQL files or even CSV files? I think there are quite still many people using MySQL and prefer these format instead of XML (as you know XML generation and parsing really take time...) Howard ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] English Wikipedia brief outage today
Hi! > Would you mind giving us a few details about the whys, hows, and > solutions? I've had a couple out of memory problems recently > (especially while trying to run DumpHTML) and our first try for a > fix was to check efficiency of our extensions. That is certainly > taking a while. Apparently we hit a combination of bad things. This is how I imagine what could have happened. There was huge ongoing batch job (recompression of blobs). There was some huge transaction (I didn't identify it yet, didn't look at it yet), that locked up lots of stuff, maybe it was recompression- related, maybe it was not. Crash recovery identified half a million of uncommitted row changes. MySQL internal undo segments limit has been reached (there were 1024 active transactions that modified data, the other option would be running out of 1G of transaction log space, which was unlikely) More and more hanging transactions led to more and more clients connecting Slaves reported lag (as there were no new transactions incoming) LB sent all the read queries to master Master had much more workload to do Probably it started allocating more memory (I don't see trace on ganglia's daily graph anymore though..) Kernel OOM killer jumped in LB decided that all slaves are lagged and wanted still to use master (this is new code, we should've failed gracefully and switched site to R/O instead, as master was down). So, for few minutes we showed down notice, then I switched the site readonly, and within few minutes, instead of waiting for crash recovery, I promoted another slave as master. Strange though, pity I didn't see what was happening at the beginning. Our enwiki master was up and running and kicking for nearly two years (we even had a stable master-slave relationship between db2 and db3 that took over a year ;-) Cheers, -- Domas Mituzas -- http://dammit.lt/ -- [[user:midom]] P.S. I was going from shower to bed, it was past 2am, and strange intuition asked me to check "whats up". I rarely turn on screens at this state of evening :) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Mailing lists problems
On Mon, Mar 30, 2009 at 8:38 PM, Platonides wrote: > Aryeh Gregor wrote: >> Well, inbox.org doesn't seem to have it enabled, so I guess you'd be out. :) > > So he would be just as everybody is now. Point . . . On Tue, Mar 31, 2009 at 1:04 AM, Marco Schuster wrote: > Google verifies SPF (and, for non-Google-Apps-users also has it so it > can be verified), for example. But it does not *only* accept SPF-validated mail. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] English Wikipedia brief outage today
On Tue, Mar 31, 2009 at 9:17 AM, Christensen, Courtney wrote: > Would you mind giving us a few details about the whys, hows, and solutions? > I've had a couple out of memory problems recently (especially while trying to > run DumpHTML) and our first try for a fix was to check efficiency of our > extensions. That is certainly taking a while. The OOM was on a database server, not an application server. Last I heard, there was a very big transaction that took out lots of locks, and Domas was investigating. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] English Wikipedia brief outage today
Hey Brion and Domas, Would you mind giving us a few details about the whys, hows, and solutions? I've had a couple out of memory problems recently (especially while trying to run DumpHTML) and our first try for a fix was to check efficiency of our extensions. That is certainly taking a while. Thanks! -Courtney ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] A proposal to simplify and improve footnote markup in Wikipedia
Håkon Wium Lie wrote: > In the quest to simplify and improve Wikipedia's HTML code, the turn > has come to footnotes. Here is a proposal that describes how the > number of elements needed to represent footnotes can be halved: > > http://www.princexml.com/howcome/2009/wikipedia/ref/ > > As far as I can tell, the proposed markup works in legacy browsers, > too. I like it, except adding square brackets with CSS. Uses that are the most likely not to support CSS are exactly the uses where the brackets are most needed. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] A proposal to simplify and improve footnote markup in Wikipedia
In the quest to simplify and improve Wikipedia's HTML code, the turn has come to footnotes. Here is a proposal that describes how the number of elements needed to represent footnotes can be halved: http://www.princexml.com/howcome/2009/wikipedia/ref/ As far as I can tell, the proposed markup works in legacy browsers, too. Cheers, -h&kon Håkon Wium Lie CTO °þe®ª howc...@opera.com http://people.opera.com/howcome ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
