Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
Ohi, > The time it takes to execute the code that glues together the regexps > will be insignificant compared to actually executing the regexps for any > article larger than a few hundred bytes. Well, you did an edge case - a long line. Actually, try replacing spaces with newlines, and you will get 25x cost difference ;-) > But the top speed of the parser (in bytes/seconds) will be largely unaffected. Damn! Domas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On 05/03/2011 10:10 PM, Ryan Lane wrote: > should re-license it in a more compatible license. LGPL is good, BSD > is likely better. What advantages does BSD offer over LGPL? ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 05/03/2011 07:45 PM, Ryan Lane wrote: > It's slightly more difficult, but it definitely isn't any easier. The > point here is that only having one implementation of the parser, which > can change at any time, which also defines the spec (and I use the > word spec here really loosely), is something that inhibits the ability > to share knowledge. I was thinking whether it would be possible to have two-tier parsing? Define what is valid wikitext, express it in BNF, write a parser in C and use it as a PHP extension. If the parser encounters invalid wikitext, enter the quirks mode AKA the current PHP parser. I assume that >90% of wikis' contents would be valid wikitext, and so the speedup should be significant. And if someone needs to reuse the content outside of Wikipedia, they can use >90% of the content very easily, and the rest not harder than right now. The only disadvantage that I see is that every addition to wikitext would have to be implemented in both parsers. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Zend performance (Was: WYSIWYG and parser plans)
Hi! > The discussion was concerning parser performance, the discussion was concerning overall parser performance, not just edge cases. > so a profile of only parser execution would have been most relevant. Indeed, try parsing decent articles with all their template trees. > In my profiling data, parser execution dominates, and as you can see, its > mostly regexp evaluation. Indeed, because you don't have anything what would invoke callbacks or branching in the code. > (With "php parser", I was referring to "zendparse" and > "lex_scan", which doesn't seem to use libpcre. I.e., almost all calls > to libpcre is made from the wikitext parser.) I usually don't know what "php parser" is, opcode caches have been around for past decade. Domas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 04/05/11 15:52, Andreas Jonsson wrote: > The time it takes to execute the code that glues together the regexps > will be insignificant compared to actually executing the regexps for any > article larger than a few hundred bytes. This is at least the case for > the articles are the the easiest for the core parser, which are articles > that contains no markup. The more markup the slower it will run. It is > possible that this slowdown will be lessened if compiled with HipHop. > But the top speed of the parser (in bytes/seconds) will be largely > unaffected. PHP execution dominates for real test cases, and HipHop provides a massive speedup. See the previous HipHop thread. http://lists.wikimedia.org/pipermail/wikitech-l/2011-April/052679.html Unfortunately, users refuse to write articles consisting only of hundreds of kilobytes of plain text, they keep adding references and links and things. So we don't really care about the parser's "top speed". -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
2011-05-03 13:25, Daniel Friesen skrev: > On 11-05-03 03:40 AM, Andreas Jonsson wrote: >> 2011-05-03 02:38, Chad skrev: >> [...] >>> I don't see any problem with keeping the parser in PHP, and as you point out >>> with HipHop support on the not-too-distant horizon the complaints about >>> performance with Zend will largely evaporate. >> But most of the parser's work consists of running regexp pattern >> matching over the article text, doesn't it? Regexp pattern matching are >> implemented by native functions. Does the Zend engine have a slow >> regexp implementation? I would have guessed that the main reason that >> the parser is slow is the algorithm, not its implementation. >> >> Best Regards, >> >> Andreas Jonsson > regexps might be fast, but when you have to run hundreds of them all > over the place and do stuff in-language then the language becomes the > bottleneck. > The time it takes to execute the code that glues together the regexps will be insignificant compared to actually executing the regexps for any article larger than a few hundred bytes. This is at least the case for the articles are the the easiest for the core parser, which are articles that contains no markup. The more markup the slower it will run. It is possible that this slowdown will be lessened if compiled with HipHop. But the top speed of the parser (in bytes/seconds) will be largely unaffected. /Andreas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 4 May 2011 15:16, Tim Starling wrote: > On 04/05/11 14:07, Daniel Friesen wrote: >> I'm fairly certain myself that his intention was "With HipHop support >> since the C that HipHop compiles PHP to can be extracted and re-used we >> can turn that compiled C into a C library that can be used anywhere by >> abstracting the database calls and what not out of the php version of >> the parser. And because HipHop has better performance we will no longer >> have to worry about parser abstractions slowing down the parser and as a >> result increasing the load on large websites like Wikipedia where they >> are noticeable. So that won't be in the way of adding those abstractions >> anymore." > > Yes that's right, more or less. HipHop generates C++ rather than C > though. > > Basically you would split the parser into several objects: > > * A parser in the traditional sense. > * An output callback object, which would handle generation of HTML or > PDF or syntax trees or whatever. > * A wiki environment interface object, which would handle link > existence checks, template fetching, etc. > > Then you would use HipHop to compile: > > * The new parser class. > * A few useful output classes, such as HTML. > * A stub environment class which has no dependencies on the rest of > MediaWiki. > > Then to top it off, you would add: > > * A HipHop extension which provides output and environment classes > which pass their calls through to C-style function pointers. > * A stable C ABI interface to the C++ library. > * Interfaces between various high level languages and the new C > library, such as Python, Ruby and Zend PHP. > > Doing this would leverage the MediaWiki development community and the > existing PHP codebase to provide a well-maintained, reusable reference > parser for MediaWiki wikitext. +1 This is the single most exciting news on the MediaWiki front since I started contributing to Wiktionary nine years ago (-: Andrew Dunbar (hippietrail) > -- Tim Starling > > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Zend performance (Was: WYSIWYG and parser plans)
2011-05-03 22:50, Domas Mituzas skrev: > Hi! > >> I'm not sure what you are profiling, > > Wikipedia :) > >> but when repeatingly requesting a >> preview of an article containing 20 bytes of data consisting of the >> pattern "a a a a a a " I got the below results. (The php parser doesn't >> seem to depend on perl regexps.) > > I'm sure nothing profiles better than a synthetic edge case. What do you mean > by it not depending on perl regexps? It is top symbol in your profile. > The discussion was concerning parser performance, so a profile of only parser execution would have been most relevant. In my profiling data, parser execution dominates, and as you can see, its mostly regexp evaluation. (With "php parser", I was referring to "zendparse" and "lex_scan", which doesn't seem to use libpcre. I.e., almost all calls to libpcre is made from the wikitext parser.) /Andreas >> CPU: CPU with timer interrupt, speed 0 MHz (estimated) >> Profiling through timer interrupt >> samples %app name symbol name >> 994 23.4933 libpcre.so.3.12.1/lib/libpcre.so.3.12.1 >> 545 12.8811 libphp5.so zendparse >> 369 8.7213 libphp5.so lex_scan >> 256 6.0506 libc-2.11.2.so memcpy >> 137 3.2380 libphp5.so zend_hash_find >> 135 3.1907 libphp5.so _zend_mm_alloc_canary_int >> 105 2.4817 libphp5.so __i686.get_pc_thunk.bx >> 902.1272 libphp5.so _zend_mm_free_canary_int >> 671.5835 libphp5.so zif_strtr >> 591.3945 libphp5.so zend_mm_add_to_free_list >> 481.1345 libphp5.so zend_mm_remove_from_free_list > > Domas > > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 04/05/11 14:07, Daniel Friesen wrote: > I'm fairly certain myself that his intention was "With HipHop support > since the C that HipHop compiles PHP to can be extracted and re-used we > can turn that compiled C into a C library that can be used anywhere by > abstracting the database calls and what not out of the php version of > the parser. And because HipHop has better performance we will no longer > have to worry about parser abstractions slowing down the parser and as a > result increasing the load on large websites like Wikipedia where they > are noticeable. So that won't be in the way of adding those abstractions > anymore." Yes that's right, more or less. HipHop generates C++ rather than C though. Basically you would split the parser into several objects: * A parser in the traditional sense. * An output callback object, which would handle generation of HTML or PDF or syntax trees or whatever. * A wiki environment interface object, which would handle link existence checks, template fetching, etc. Then you would use HipHop to compile: * The new parser class. * A few useful output classes, such as HTML. * A stub environment class which has no dependencies on the rest of MediaWiki. Then to top it off, you would add: * A HipHop extension which provides output and environment classes which pass their calls through to C-style function pointers. * A stable C ABI interface to the C++ library. * Interfaces between various high level languages and the new C library, such as Python, Ruby and Zend PHP. Doing this would leverage the MediaWiki development community and the existing PHP codebase to provide a well-maintained, reusable reference parser for MediaWiki wikitext. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
- Original Message - > From: "Tim Starling" > I wasn't saying that the current MediaWiki parser is suitable for > reuse, I was saying that it may be possible to develop the MediaWiki > parser into something which is reusable. Aren't there a couple of parsers already which claim 99% compliance or better? Did anything ever come of trying to assemble a validation suite, All Those Years Ago? Or, alternatively, deciding how many pages it's acceptable to break in the definition of a formal spec? Cheers, -- jra ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
- Original Message - > From: "Daniel Friesen" > I'm fairly certain myself that his intention was "With HipHop support > since the C that HipHop compiles PHP to can be extracted and re-used > we can turn that compiled C into a C library that can be used anywhere by > abstracting the database calls and what not out of the php version of > the parser. And because HipHop has better performance we will no > longer have to worry about parser abstractions slowing down the parser and as > a result increasing the load on large websites like Wikipedia where they > are noticeable. So that won't be in the way of adding those > abstractions anymore." What I get for not paying any attention to Facebook Engineering. *That's* what HipHop does? > Naturally of course if it's a C library you can build at least an > extension/plugin for a number of languages. You would of course have > to install the ext/plug though so it's not a shared-hosting thing. True. But that's still a derivative work. And from experience, I can tell you that you *don't* want to work with the *output* of a code generator/cross-compiler. Cheers, -- jra ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 11-05-03 08:46 PM, Jay Ashworth wrote: > - Original Message - >> From: "MZMcBride" >>> Now that we have HipHop support, we have the ability to turn >>> MediaWiki's core parser into a fast, reusable library. The performance >>> reasons for limiting the amount of abstraction in the core parser will >>> disappear. How many wikitext parsers does the world really need? >> I realize you have a dry wit, but I imagine this joke was lost on >> nearly everyone. You're not really suggesting that everyone who wants to >> parse MediaWiki wikitext compile and run HipHop PHP in order to do so. > I'm fairly certain that his intention was "If the parser is HipHop compliant, > then the performance improvements that will realize for those who need them > will obviate the need to rewrite the parser in anything, while those who > run small enough wikiae not to care, won't need to care." > > That does *not*, of course, answer the "if you don't have more than one > compliant parser, then the code is part of your formal spec, and you > *will* get bitten eventually" problem. > > Of course, Mediawiki's parser has *three* specs: whatever formal one > has been ginned up, finally; the code; *and* 8 or 9 GB of MWtext on the > Wikipedias. > > Cheers, > -- jra I'm fairly certain myself that his intention was "With HipHop support since the C that HipHop compiles PHP to can be extracted and re-used we can turn that compiled C into a C library that can be used anywhere by abstracting the database calls and what not out of the php version of the parser. And because HipHop has better performance we will no longer have to worry about parser abstractions slowing down the parser and as a result increasing the load on large websites like Wikipedia where they are noticeable. So that won't be in the way of adding those abstractions anymore." Naturally of course if it's a C library you can build at least an extension/plugin for a number of languages. You would of course have to install the ext/plug though so it's not a shared-hosting thing. ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
- Original Message - > From: "Peter Youngmeister" > If you guys still think about ideas as things that can be stolen, > perhaps you should check out the open source movement. Here's a good > reference: > > http://en.wikipedia.org/wiki/Open_source Aw, c'mon, Peter. No strawmen; it's late. The reasons why many programmers prefer GPL to BSD -- to keep the work they've invested long hours in for free from being submerged in someone's commercial project with no recompense to them -- which GPL forbids and BSD does not -- is widely understoood. Myself, I'm firmly convinced after 30 years in this business, that, for all its faults, the choice of the GPL by Linus changed the face of computing (and damned near everything else) just a much as the Apollo project's investment in microelectronics gave us all PCs to run them on in the first place. You're welcome to disagree (though not on this list; there are lots of places better suited to license advocacy), but it would probably be good not to scoff at people for holding that view. 'Specially when you're using their code. :-) Cheers, -- jra ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
- Original Message - > From: "Neil Harris" > I think it cannot be emphasized enough that what's valuable about > Wikipedia and other similar wikis is the hard-won _content_, not the > software used to write and display it at any given, which is merely a > means to that end. > > Fashions in programming languages and data formats come and go, but > the person-centuries of writing effort already embodied in Mediawiki's > wikitext format needs to have a much longer lifespan: having a > well-defined syntax for its current wikitext format will allow the > content itself to continue to be maintained for the long term, beyond > the restrictions of its current software or encoding format. The project of creating a formal specification for Mediawikitext was one of the primary reasons for the creation of the (largely dormant) wikitext-l list. I fell off shortly after it was created myself, so I don't know how far along that project got -- except that I know that it was decided that since MWtext was not -- and could not be -- a strict subset of Creole, that Creole was a Pretty Nice Idea... and we had no time for it. For my money, that means the Creole folks lost[1], but what do I know. Cheers, -- jra [1] Which is not incompatible with observations then that MWtext has some really unaccepable boners in itself... ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
- Original Message - > From: "MZMcBride" > > Now that we have HipHop support, we have the ability to turn > > MediaWiki's core parser into a fast, reusable library. The performance > > reasons for limiting the amount of abstraction in the core parser will > > disappear. How many wikitext parsers does the world really need? > > I realize you have a dry wit, but I imagine this joke was lost on > nearly everyone. You're not really suggesting that everyone who wants to > parse MediaWiki wikitext compile and run HipHop PHP in order to do so. I'm fairly certain that his intention was "If the parser is HipHop compliant, then the performance improvements that will realize for those who need them will obviate the need to rewrite the parser in anything, while those who run small enough wikiae not to care, won't need to care." That does *not*, of course, answer the "if you don't have more than one compliant parser, then the code is part of your formal spec, and you *will* get bitten eventually" problem. Of course, Mediawiki's parser has *three* specs: whatever formal one has been ginned up, finally; the code; *and* 8 or 9 GB of MWtext on the Wikipedias. Cheers, -- jra ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87363]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87363. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87363#c0 Commit summary: removing extra ?s ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87359]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87359. Old Status: fixme New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c0 Commit summary: new subheader messages ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87382]: New comment added
User "Dantman" posted a comment on MediaWiki.r87382. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87382#c16536 Commit summary: Add proper $out setting to Skin::getCategories which is present in the other similar methods. Replaces r87380. Comment: FWIW, I plan to kill the $out args I added and replace them with that `$out = $this->getContext()->output`. And go the other direction, depreciating $out args in most of the methods that originally had them (with the exception of Skin::outputPage which is the entrypoint that may set the context). ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87093]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87093. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87093#c0 Commit summary: removing FlickrChecker for now ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87371]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87371. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87371#c0 Commit summary: setting links to open in new windows, per r87359 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87366]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87366. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87366#c0 Commit summary: better api calls for flickr, fixing attribution license, restoring messages lost in merge ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87369]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87369. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87369#c0 Commit summary: more docs for the various methods you can use ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87133]: Revision status changed
User "^demon" changed the status of MediaWiki.r87133. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87133#c0 Commit summary: added since tag for people that care about compat ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87317]: Revision status changed
User "Trevor Parscal" changed the status of MediaWiki.r87317. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87317#c0 Commit summary: Removed unused messages: 'extremelycomplextest' 'internallinktest' 'linktest' 'magictest' 'mwe-copyright-custom' 'mwe-copyright-macro' 'mwe-loading-upwiz' 'mwe-upwiz-about-format' 'mwe-upwiz-about-this-work' 'mwe-upwiz-add-file-0' 'mwe-upwiz-browse' 'mwe-upwiz-categories-another' 'mwe-upwiz-categories-intro' 'mwe-upwiz-change' 'mwe-upwiz-click-here' 'mwe-upwiz-desc-add-0' 'mwe-upwiz-editing' 'mwe-upwiz-file-need-complete' 'mwe-upwiz-file-need-start' 'mwe-upwiz-filename-tag' 'mwe-upwiz-license' 'mwe-upwiz-license-incompatible-cc' 'mwe-upwiz-license-incompatible-pd' 'mwe-upwiz-license-show-all-any-license' 'mwe-upwiz-macro-edit' 'mwe-upwiz-macro-edit-intro' 'mwe-upwiz-other-prefill' 'mwe-upwiz-showall' 'mwe-upwiz-thanks-link' 'namespacedtest' 'pluraltest' ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87219]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87219. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87219#c0 Commit summary: Trailing whitespace removed. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87258]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87258. Old Status: fixme New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87258#c0 Commit summary: FlickrChecker fixes, follow-up to r87002 and r87031 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87096]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87096. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87096#c0 Commit summary: follow-up to 87002, take into account Flickr adding new licenses, i.e. unknown license ID ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87043]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87043. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87043#c0 Commit summary: fixing sloppy var scoping from r87002 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87095]: New comment added, and revision status changed
User "NeilK" changed the status of MediaWiki.r87095. Old Status: new New Status: ok User "NeilK" also posted a comment on MediaWiki.r87095. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87095#c16535 Commit summary: fixing formatting bug in Chrome caused by long error messages - no longer tries to force everything onto one line Comment: I was surprised the change from span to div worked, since this widget is used in a lot of other places. But... okay ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87218]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87218. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87218#c0 Commit summary: Use a verb in 'mwe-upwiz-api-error-duplicate-archive-popup-title' ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87094]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87094. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87094#c0 Commit summary: punctuation for sentences ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On 4 May 2011 08:19, Krinkle wrote:
> Op 3 mei 2011, om 22:56 heeft Ryan Lane het volgende geschreven:
>
>> On Tue, May 3, 2011 at 1:33 PM, Trevor Parscal
>> wrote:
>>> On it's own, it would be essentially useless.
>>>
>>
>> The parser has a configuration state, takes wikitext in, and gives
>> back html. It pulls additional data from the database in these steps
>> as well, yes. However, I don't see how this would be different than
>> any other implementation of the parser. All implementations will
>> require configuration state, and will need to deal with things like
>> templates and extensions.
>>
>> Though I prefer the concept of alternative parsers (for all the
>> reasons mentioned in the other threads), I do think having our
>> reference implementation available as a library is a good concept. I
>> feel that making it available in a suitable license is ideal.
>>
>> - Ryan
>>
>
> Afaik parser does not need a database or extension hooks for minimum but
> fully operational use.
>
> {{unknown templates}} default to redlinks,
> {{int:messages}} default to ,
> and {{#functions}} default to literals,
> {{MAGICWORDS}} to red links,
> etc...
>
> If a user of the parser would not have any of these (either none
> existing or no
> registry / database configured at all). It would fallback to the
> behaviour as if
> they are inexistant, not a problem ?
I agree a parser would not need a database but it would need a
standard interface or abstraction that in the full MediaWiki would
call to the database. Offline readers would implement this interface
to extract the wikitext from their compressed format or direct from an
XML dump file.
Some datamining tools might just stub this interface and deal with the
bare minimum.
Extension hooks are more interesting. I might assume offline readers
want as close results to the official sites as possible so will want
to implement the same hooks.
Other non-wikitext or non-page data from the database would also go
into the same interface/abstraction, or a separate one.
Andrew Dunbar (hippietrail)
> By having this available as a parser sites that host blogs and forums
> could potentially use wikitext to format their comments and forum
> threads
> (to avoid visitors from having to for example learn Wikitext for their
> wiki,
> WYSIWYM WYMeditor for WordPress and BBCode for a forum).
>
> Instead they could all be the same syntax. And within wiki context
> magic words, extensions, int messages etc. would be fed from the wiki
> database,
> outside just static.
>
> --
> Krinkle
>
>
>
>
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87097]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87097. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87097#c0 Commit summary: better fix for bug 26178 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87359]: New comment added
User "Kaldari" posted a comment on MediaWiki.r87359. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c16534 Commit summary: new subheader messages Comment: fixed in r87371. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87318]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87318. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87318#c0 Commit summary: followup to 87313, forgot to add published message to hooks ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On 4 May 2011 06:33, Trevor Parscal wrote: > I think the idea that we might break the existing PHP "parser" out into a > library for general use is rather silly. > > The "parser" is not a parser, it's a macro expander with a pile of > regular-expressions used to convert short-hand HTML into actual HTML. The Oh don't be silly. It may not be an LALR(1) parser or an LL parser or even a recursive descent parser but last I checked parsing was the act of breaking down a text into its elements, which the parser does. It just does it in a pretty clunky way. Whether it stores the results in an AST or in bunches of random state all over the place doesn't mean it's doing something other than parsing. A more accurate argument is that it's not just a parser since goes directly on to transforming the input into HTML, which is the equivalent of code generation. > code that it outputs is highly dependent on the state of the wiki's > configuration and database content at the moment of "parsing". It also is > useless to anyone wanting to do anything other than render a page into HTML, > because the output is completely opaque as to where any of it > was derived. Dividing the "parser" off into a library would require an > substantial amount of MediaWiki code to be ported too just to get it > working. On it's own, it would be essentially useless. It seems we're getting bogged won in semantics because in MediaWiki we use the word "parser" in two incompatible ways. 1) The PHP classes which convert wikitext to HTML 2) A hypothetical or postulated part of MediaWiki which does not exist to generate an intermediate form (AST) between wikitext and HTML. So the first thing we need to do is decide which of these two concepts of parser we're talking about. Would it be useful to have a library that can convert wikitext to HTML? Yes. Would it be useful to have a library that can convert wikitext to an AST? Unclear. Would it be useful to have a library that can convert such AST to HTML? Because of the semantic soup nobod has even brought this up yet. > So, it's probably not an issue what license this hypothetical code would be > released under. > > - Trevor I'm pretty sure the offline wikitext parsing community would care about the licensing as a separate issue to what kind of parser technology it uses internally. Andrew Dunbar (hippietrail) > On Tue, May 3, 2011 at 1:25 PM, David Gerard wrote: > >> On 3 May 2011 21:15, Domas Mituzas wrote: >> >> >> Thoughts? Also, for re-licensing, what level of approval do we need? >> >> All authors of the parser, or the current people in an svn blame? >> >> > Current people are doing 'derivative work' on previous authors work. I >> think all are needed. Pain oh pain. >> >> >> This is the other reason to reduce it to mathematics, which can then >> be freely reimplemented. >> >> >> - d. >> >> ___ >> Wikitech-l mailing list >> Wikitech-l@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l >> > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87314]: Revision status changed
User "Trevor Parscal" changed the status of MediaWiki.r87314. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87314#c0 Commit summary: retry thumbnails for several seconds before declaring failure. All similar thumbnails in layout "subscribe" interest, and then the thumbnail is "published" to them when ready. Should address bug 28782 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
> Personally, I don't see any problem with a parser library being GPL. > You can still link it with proprietary code as long as you don't > distribute the result, so it would be fine for research projects or > similar that rely on proprietary components. You can always *use* > GPLd code however you like. If you want to *distribute* proprietary > (or otherwise GPL-incompatible) code that depends on my volunteer > contributions, I'm happy to tell you to go jump off a bridge. > You'd have an issue with a proprietary application using the wikitext parser as a library? You really find the LGPL completely unacceptable in this situation? Seems like kind of a hardline position to take. That same application could make API calls to MediaWiki, using it in essentially the same way, without the license restrictions. Also, GPL, in our use case, is fairly ineffective. Even if an application makes PHP calls directly into MediaWiki, that application doesn't necessarily need to be GPL, since there is no actual linking occurring. Not all MediaWiki extensions are GPL, for instance. Essentially, this will just limit a C version of the software, which is slightly lame. Meh. If we have a GPL library, I'll just wrap it in a wsgi python library to act as a shim. - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87359]: New comment added
User "NeilK" posted a comment on MediaWiki.r87359.
Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c16533
Commit summary:
new subheader messages
Comment:
you can do it like this -- pass in a jQuery object instead of a string URL.
Magic. :)
.msg(
'mwe-upwiz-subhead-bugs',
$j( '' ).attr( { href: mw.UploadWizard.config['bugList'], target:
'_blank' } )
);
or, find the 's after you do all .msg() and use jquery to add a target attr
to them all
___
MediaWiki-CodeReview mailing list
mediawiki-coderev...@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87312]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87312. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87312#c0 Commit summary: use "donate" message rather than "upload" for big button ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87359]: New comment added
User "NeilK" posted a comment on MediaWiki.r87359. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c16532 Commit summary: new subheader messages Comment: looks good, just missing target=_blank? ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87367]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87367. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87367#c0 Commit summary: adding missing tests, follow-up to r87359 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87370]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87370. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87370#c0 Commit summary: a simpler way to fix bug 26178 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 04/05/11 06:38, Neil Kandalgaonkar wrote: > On 5/2/11 5:28 PM, Tim Starling wrote: >> How many wikitext parsers does the world really need? > > That's a tricky question. What MediaWiki calls parsing, the rest of the > world calls > > 1. Parsing > 2. Expansion (i.e. templates, magic) > 3. Applying local state, preferences, context (i.e. $n, prefs) > 4. Emitting > > And phases 2 and 3 depend heavily upon the state of the local wiki at > the time the parse is requested. If you've ever tried to set up a test > wiki that works like Wikipedia or Wikimedia Commons you'll know what I'm > talking about. I wasn't saying that the current MediaWiki parser is suitable for reuse, I was saying that it may be possible to develop the MediaWiki parser into something which is reusable. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87368]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87368. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87368#c0 Commit summary: fixing bugzilla link, follow-up to r87359 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87156]: New comment added, and revision status changed
User "^demon" changed the status of MediaWiki.r87156. Old Status: new New Status: ok User "^demon" also posted a comment on MediaWiki.r87156. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87156#c16531 Commit summary: quick and dirty language stats Comment: Should remove svn:executable prop. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r86664]: Revision status changed
User "^demon" changed the status of MediaWiki.r86664. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/86664#c0 Commit summary: fix off-by-one in month when defaulting to now ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87355]: Revision status changed
User "^demon" changed the status of MediaWiki.r87355. Old Status: new New Status: deferred Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87355#c0 Commit summary: Localisation updates for core and extension messages from translatewiki.net (2011-05-02 20:13:00 UTC) ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Tue, May 3, 2011 at 6:56 PM, Aryeh Gregor wrote: > On Tue, May 3, 2011 at 6:16 PM, Chad wrote: >> Who do we consider significant? Would it be possible to get >> consensus on a relicensing? > > As far as I know, the way the GPL works makes it effectively > impossible to relicense a large project to something more permissive. > You'd have to get permission from literally everyone who made > nontrivial contributions, or else rewrite their code. But if there's > serious interest in this, someone should get an official opinion from > Wikimedia's lawyers on how (or if) it could be done. > > Personally, I don't see any problem with a parser library being GPL. > You can still link it with proprietary code as long as you don't > distribute the result, so it would be fine for research projects or > similar that rely on proprietary components. You can always *use* > GPLd code however you like. If you want to *distribute* proprietary > (or otherwise GPL-incompatible) code that depends on my volunteer > contributions, I'm happy to tell you to go jump off a bridge. > I was just speculating. I don't have any problems with the GPL :) -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87359]: New comment added
User "Kaldari" posted a comment on MediaWiki.r87359. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c16530 Commit summary: new subheader messages Comment: Oops, I meant to do both tests actually (like I do with feedback). Fixed in r87367. Bugzilla link fixed in r87368. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r86041]: New comment added
User "^demon" posted a comment on MediaWiki.r86041. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/86041#c16529 Commit summary: r86001, now with less scariness :P I took out the delete action and did purge instead, which is a much more self-contained action-with-a-form. Also implement a few changes suggested by Brion on IRC last night. Comment: I don't like introducing a new global for this. Move it into the class. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r86041]: New comment added
User "Happy-melon" posted a comment on MediaWiki.r86041. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/86041#c16528 Commit summary: r86001, now with less scariness :P I took out the delete action and did purge instead, which is a much more self-contained action-with-a-form. Also implement a few changes suggested by Brion on IRC last night. Comment: There aren't fatals as a result because Action::factory() loads Action.php, but you're right that they should have entries regardless. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87359]: New comment added
User "NeilK" posted a comment on MediaWiki.r87359. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c16527 Commit summary: new subheader messages Comment: uh, if it's not obvious, there are a lot of '' which got turned into italicization in my previous comment ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87359]: New comment added, and revision status changed
User "NeilK" changed the status of MediaWiki.r87359. Old Status: new New Status: fixme User "NeilK" also posted a comment on MediaWiki.r87359. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87359#c16526 Commit summary: new subheader messages Comment: 1) The tests for ( mw.UploadWizard.config['something'] !== '' ) are doing the wrong thing. If those items are missing, the value returned is undefined, which is also !== '' You want to use typeof (mw.UploadWizard.config['something']) !== undefined or perhaps mw.isDefined() 2) The URL for bugzilla doesn't work, those saved searches are specific to particular users This will work better, I think. Be careful of those list_id arguments, they refer to ephemeral cached listings. https://bugzilla.wikimedia.org/buglist.cgi?query_format=advanced&component=UploadWizard&resolution=---&product=MediaWiki+extensions 3) could you add target=_blank to all the links, so they don't interrupt an upload? Thanks. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87258]: New comment added
User "Kaldari" posted a comment on MediaWiki.r87258. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87258#c16525 Commit summary: FlickrChecker fixes, follow-up to r87002 and r87031 Comment: Removed the extra question marks. The 'jsoncallback=?' was to generate a random callback function name (normal jsonp methodology). I didn't know you could suppress the callback function entirely. That's much more sensible. Implemented in r87366. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Tue, May 3, 2011 at 6:16 PM, Chad wrote: > Who do we consider significant? Would it be possible to get > consensus on a relicensing? As far as I know, the way the GPL works makes it effectively impossible to relicense a large project to something more permissive. You'd have to get permission from literally everyone who made nontrivial contributions, or else rewrite their code. But if there's serious interest in this, someone should get an official opinion from Wikimedia's lawyers on how (or if) it could be done. Personally, I don't see any problem with a parser library being GPL. You can still link it with proprietary code as long as you don't distribute the result, so it would be fine for research projects or similar that rely on proprietary components. You can always *use* GPLd code however you like. If you want to *distribute* proprietary (or otherwise GPL-incompatible) code that depends on my volunteer contributions, I'm happy to tell you to go jump off a bridge. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] auditcode.py - discern class structure
Paul Houle wrote: > Note that there is a PHP tokenizer built into PHP which makes it > straightforward to develop tools like this in PHP: > > http://php.net/manual/en/book.tokenizer.php > > A practical example can be found here > > http://gen5.info/q/2009/01/09/an-awesome-autoloader-for-php > > Now, it would be nice to have a parser available that can see the > tree structure. The tokenizer is slow. It's not worth taking it out for this. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] What is wrong with Wikia's WYSIWYG?
Some of them are already going. After talking with some of them about our new parser and editor plans, and that we would be doing work on that in Berlin, they said they were likely to send some people. Obviously it's up to a variety of factors how many of Wikia's people show up, but we do know that they are aware of the topics, the relevance of their work and what will be done at the conference, and have shown interest. Historically, Wikia sends a few people each year, and often presents on something they are working on. - Trevor On Mon, May 2, 2011 at 5:26 PM, Sumana Harihareswara wrote: > On 05/02/2011 06:40 PM, Owen Davis wrote: > > I think the current plan is to see what transpires with the > > upcoming Parser redesign and keep our code in sync. The > > primary authors of this RTE reskin will be at the Berlin > > hackathon as well. > > Who are those authors? I'd like to know so I can find them in Berlin > and drag them into any relevant discussions, etc. > > -Sumana Harihareswara > > > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] auditcode.py - discern class structure
Russell N. Nelson - rnnelson wrote:
> Maybe there's a better tool to tell you what function is
>defined in what class in PHP, but I couldn't find one in
>the time it would take me to write it, so I wrote it.
>It's not even a screenful. Give it the class definitions,
>in class hierarchy order, on the command line.
That restriction isn't too friendly.
>It will pull out the class name and function names, and tell
>you which function is actually being implemented by which class.
>It doesn't pay any attention to parent::self() calls, so
>you should watch out for them.
> -russ
> #!/usr/bin/python
>
> import sys, re
>
> functions = {}
> classes = {}
> cl = None
> for fn in sys.argv[1:]:
> for line in open(fn):
> match =
> re.match(r'(abstract\s+)?class\s+(.*?)\s+(extends\s+(.*?)\s+)?\{', line)
> if match:
> cl = match.group(2)
> supercl = match.group(4)
> classes[cl] = supercl
> if supercl:
> classes[supercl] # superclass should already be exist
Also note, some files include both the class and the superclass, so
"give in class hierarchy order" may not be possible (I don't know if
there's any file that includes the child before the parent, though).
> match = re.match(r'\s*(public\s+|static\s+)?function\s+(.*?)\(', line)
I understand you may not want protected or private functions, but
this regex will also miss the 536 functions that are public static, and
the 7 that are static public.
> if match:
> # we have a function definition.
> funct = match.group(2)
> functions[funct] = cl
>
> keys = functions.keys()
> keys.sort()
> for key in keys:
> print key,functions[key]
If you are adventurous, you may want to add an option to create a file
similar to parent.classes in check-vars.php Functions are already being
tracked, so it shouldn't be too hard.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r86041]: New comment added, and revision status changed
User "Platonides" changed the status of MediaWiki.r86041. Old Status: new New Status: fixme User "Platonides" also posted a comment on MediaWiki.r86041. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/86041#c16524 Commit summary: r86001, now with less scariness :P I took out the delete action and did purge instead, which is a much more self-contained action-with-a-form. Also implement a few changes suggested by Brion on IRC last night. Comment: You are freely using FormAction and FormlessAction but they aren't registered in the autoloader. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
Op 3 mei 2011, om 22:56 heeft Ryan Lane het volgende geschreven:
> On Tue, May 3, 2011 at 1:33 PM, Trevor Parscal
> wrote:
>> On it's own, it would be essentially useless.
>>
>
> The parser has a configuration state, takes wikitext in, and gives
> back html. It pulls additional data from the database in these steps
> as well, yes. However, I don't see how this would be different than
> any other implementation of the parser. All implementations will
> require configuration state, and will need to deal with things like
> templates and extensions.
>
> Though I prefer the concept of alternative parsers (for all the
> reasons mentioned in the other threads), I do think having our
> reference implementation available as a library is a good concept. I
> feel that making it available in a suitable license is ideal.
>
> - Ryan
>
Afaik parser does not need a database or extension hooks for minimum but
fully operational use.
{{unknown templates}} default to redlinks,
{{int:messages}} default to ,
and {{#functions}} default to literals,
{{MAGICWORDS}} to red links,
etc...
If a user of the parser would not have any of these (either none
existing or no
registry / database configured at all). It would fallback to the
behaviour as if
they are inexistant, not a problem ?
By having this available as a parser sites that host blogs and forums
could potentially use wikitext to format their comments and forum
threads
(to avoid visitors from having to for example learn Wikitext for their
wiki,
WYSIWYM WYMeditor for WordPress and BBCode for a forum).
Instead they could all be the same syntax. And within wiki context
magic words, extensions, int messages etc. would be fed from the wiki
database,
outside just static.
--
Krinkle
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r86990]: Revision status changed
User "NeilK" changed the status of MediaWiki.r86990. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/86990#c0 Commit summary: titles on the inputs are no longer necessary. these were left over from the old tipsy scheme ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Tue, May 3, 2011 at 6:14 PM, Aryeh Gregor wrote: > On Tue, May 3, 2011 at 2:11 PM, Domas Mituzas wrote: >> Which of course allows me to fork the thread and ask why does MediaWiki have >> to be GPL licensed. > > Because all it takes is one developer with substantial contributions > who doesn't want to relicense, and then you have to rewrite all their > contributions and everything based on their contributions if you want > to change the license. That's what a viral license is, after all. Of > course, an independent component could be non-GPL-licensed, if it was > written from scratch. > Who do we consider significant? Would it be possible to get consensus on a relicensing? -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87361]: Revision status changed
User "Reedy" changed the status of MediaWiki.r87361. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87361#c0 Commit summary: Use canonical case ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Tue, May 3, 2011 at 2:11 PM, Domas Mituzas wrote: > Which of course allows me to fork the thread and ask why does MediaWiki have > to be GPL licensed. Because all it takes is one developer with substantial contributions who doesn't want to relicense, and then you have to rewrite all their contributions and everything based on their contributions if you want to change the license. That's what a viral license is, after all. Of course, an independent component could be non-GPL-licensed, if it was written from scratch. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87258]: New comment added
User "NeilK" posted a comment on MediaWiki.r87258.
Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87258#c16523
Commit summary:
FlickrChecker fixes, follow-up to r87002 and r87031
Comment:
The arguments for getJSON are strange. Are you sure this works? I couldn't get
it to do anything.
I'm not sure what you're trying to do with the '?jsoncallback=?', especially
since you define apiUrl as already having a trailing '?' elsewhere. So doesn't
that make '??jsoncallback=?' ?
If you just don't want a JSON callback, you can use nojsoncallback=1. This
worked for me:
this.apiUrl = 'http://api.flickr.com/services/rest';
apiKey = 'blah blah key here';
$.getJSON( apiUrl, { nojsoncallback: 1, format : 'json', api_key: apiKey,
method : 'flickr.panda.getList' } );
___
MediaWiki-CodeReview mailing list
mediawiki-coderev...@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] auditcode.py - discern class structure
On Tue, May 3, 2011 at 10:25 AM, Paul Houle wrote: > Note that there is a PHP tokenizer built into PHP which makes it > straightforward to develop tools like this in PHP: > > http://php.net/manual/en/book.tokenizer.php > > A practical example can be found here > > http://gen5.info/q/2009/01/09/an-awesome-autoloader-for-php We have a practical example too: http://svn.wikimedia.org/viewvc/mediawiki/trunk/tools/code-utils/stylize.php?view=markup ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87269]: Revision status changed
User "NeilK" changed the status of MediaWiki.r87269. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87269#c0 Commit summary: fixing closure for jQuery, follow-up for r87258 ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Tue, May 3, 2011 at 10:56 PM, Ryan Lane wrote: > The parser has a configuration state, takes wikitext in, and gives > back html. It pulls additional data from the database in these steps > as well, yes. However, I don't see how this would be different than > any other implementation of the parser. All implementations will > require configuration state, and will need to deal with things like > templates and extensions. > Not all implementations will want to output HTML, though. Like Neil said in the other thread, some implementations will want to output other formats (HTML for mobile, or PDF) or just want to analyze stuff (metadata from infoboxes/templates for Google or OpenStreetMap). What we have right now is mostly (the preprocessor is nicely separate now, but still) a black box that eats wikitext, reads additional data from places, and spits out HTML. A truly reusable component would at least produce something like an abstract syntax tree that can be rendered or traversed by different consumers to produce different results. Reducing the external dependencies is hard, I agree with that part. However, some components of the (hypothetically broken-up) parser don't necessarily need to know as much, so some gains could possibly be made there. Roan Kattouw (Catrope) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r86473]: Revision status changed
User "DieBuche" changed the status of MediaWiki.r86473. Old Status: new New Status: reverted Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/86473#c0 Commit summary: Fix Bug 28537: JUI buttons now have black as text-color ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87351]: New comment added
User "RussNelson" posted a comment on MediaWiki.r87351. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87351#c16522 Commit summary: Only one extension was using getThumbnail Comment: Argh, yeah. I put in tabs, which vim then ``helpfully'' converted to spaces ... but just on that one line. I have to figure out how to convince it to only smash tabs when I'm editing a .py file. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Tue, May 3, 2011 at 1:33 PM, Trevor Parscal wrote: > I think the idea that we might break the existing PHP "parser" out into a > library for general use is rather silly. > Well, if that's the case, why was it brought up in the discussion to begin with? Here's the comment Tim made: "Now that we have HipHop support, we have the ability to turn MediaWiki's core parser into a fast, reusable library. The performance reasons for limiting the amount of abstraction in the core parser will disappear. How many wikitext parsers does the world really need?" He tends to think it's an option. Domas mentioned in IRC that he made a standalone version of the parser a while back, as well. > The "parser" is not a parser, it's a macro expander with a pile of > regular-expressions used to convert short-hand HTML into actual HTML. The > code that it outputs is highly dependent on the state of the wiki's > configuration and database content at the moment of "parsing". It also is > useless to anyone wanting to do anything other than render a page into HTML, > because the output is completely opaque as to where any of it > was derived. Dividing the "parser" off into a library would require an > substantial amount of MediaWiki code to be ported too just to get it > working. On it's own, it would be essentially useless. > The parser has a configuration state, takes wikitext in, and gives back html. It pulls additional data from the database in these steps as well, yes. However, I don't see how this would be different than any other implementation of the parser. All implementations will require configuration state, and will need to deal with things like templates and extensions. Though I prefer the concept of alternative parsers (for all the reasons mentioned in the other threads), I do think having our reference implementation available as a library is a good concept. I feel that making it available in a suitable license is ideal. - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Bugzilla Weekly Report
On 05/03/2011 12:43 PM, quim@nokia.com wrote: > Fwiw a MeeGo contributor has created a script to retrieve weekly information > from http://bugs.meego.com and make these nice and useful reports: > > http://www.octofish.net/meegobugjar/platform/ > > Let me know if you are interested in an equivalent for bugzilla.wikimedia.org > > -- > Quim Quim, it has SPARKLINES, how can we resist?! ;-) Yes, I am also interested in an equivalent for our Bugzilla. -Sumana ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Zend performance (Was: WYSIWYG and parser plans)
Hi! > I'm not sure what you are profiling, Wikipedia :) > but when repeatingly requesting a > preview of an article containing 20 bytes of data consisting of the > pattern "a a a a a a " I got the below results. (The php parser doesn't > seem to depend on perl regexps.) I'm sure nothing profiles better than a synthetic edge case. What do you mean by it not depending on perl regexps? It is top symbol in your profile. > CPU: CPU with timer interrupt, speed 0 MHz (estimated) > Profiling through timer interrupt > samples %app name symbol name > 994 23.4933 libpcre.so.3.12.1/lib/libpcre.so.3.12.1 > 545 12.8811 libphp5.so zendparse > 369 8.7213 libphp5.so lex_scan > 256 6.0506 libc-2.11.2.so memcpy > 137 3.2380 libphp5.so zend_hash_find > 135 3.1907 libphp5.so _zend_mm_alloc_canary_int > 105 2.4817 libphp5.so __i686.get_pc_thunk.bx > 902.1272 libphp5.so _zend_mm_free_canary_int > 671.5835 libphp5.so zif_strtr > 591.3945 libphp5.so zend_mm_add_to_free_list > 481.1345 libphp5.so zend_mm_remove_from_free_list Domas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87354]: Revision status changed
User "Jack Phoenix" changed the status of MediaWiki.r87354. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87354#c0 Commit summary: remove double "for" ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 5/2/11 5:28 PM, Tim Starling wrote: >How many wikitext parsers does the world really need? That's a tricky question. What MediaWiki calls parsing, the rest of the world calls 1. Parsing 2. Expansion (i.e. templates, magic) 3. Applying local state, preferences, context (i.e. $n, prefs) 4. Emitting And phases 2 and 3 depend heavily upon the state of the local wiki at the time the parse is requested. If you've ever tried to set up a test wiki that works like Wikipedia or Wikimedia Commons you'll know what I'm talking about. As for whether the rest of the world needs another wikitext parser: well, they keep writing them, so there must be some reason why this keeps happening. It's true that language chauvinism plays a part, but the inflexibility of the current approach is probably a big factor as well. The current system mashes parsing and emitting to HTML together, very intimately, and a lot of people would like those to be separate. - if they're doing research or stats, and want a more "pure", more normalized form than HTML or Wikitext. - if they're Google, and they want to get all the city infobox data and reuse it (this is a real request we've gotten) - if they're OpenStreetMaps, and the same thing; - if they're emitting to a different format (PDF, LaTeX, books); - if they're emitting to HTML but with different needs (like mobile); And then there's the stuff which you didn't know you wanted, but which becomes easy once you have a more flexible parser. A couple of months ago I wrote a mini PEG-based wikitext parser in JavaScript, that Special:UploadWizard is using, today, live on Commons. http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/UploadWizard/resources/mediawiki.language.parser.js?view=markup http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/UploadWizard/resources/mediawiki.language.parser.peg?view=markup While it was a bit of a heavy download (7K compressed) this gave me the ability to do pluralizations in the frontend (e.g. "3 out of 5 uploads complete") even for difficult languages like Arabic. Great! But the unexpected benefit was that it also made it a snap to add very complicated interface behaviour to our message strings. Actually, right now, with this library + the ingenious way that wikitext does i18n, we may have one of the best libraries out there for internationalized user interfaces. I'm considering splitting it off; it could be useful for any project that used translatewiki. But I don't actually want to use JavaScript for anything but the final rendering stages (I'd rather move most of this parser to PHP) so stay tuned. Anyway, I think it's obviously possible for us to do some RTE, and some of this stuff, with the current parser. But I'm optimistic that a new parsing strategy will be a huge benefit to our community, and our partners, and partners we didn't even know we could have. Imagine doing RTE with an implementation in a JS frontend, that is generated from some of the same sources that the PHP backend uses. For what it's worth: whenever I meet with Wikia employees the topic is always about what MediaWiki and the WMF can do to make their RTE hacks obsolete. That doesn't mean that their RTE isn't the right way forward, but the people who wrote it don't seem to be very strong advocates for it. But I don't want to put words in their mouth; maybe one of them can add more to this thread? -- Neil Kandalgaonkar ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] WorkingWiki extension
Just added to MediaWiki.org: http://www.mediawiki.org/wiki/Extension:WorkingWiki WorkingWiki is a software extension for MediaWiki that makes a wiki into a powerful environment for collaborating on publication-quality manuscripts and software projects. It's designed for research labs' wikis, but may have diverse other uses as well. (I probably should have made it public long ago, but there was a big refactor and it took a long time to settle out...) Lee Worden McMaster University ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Release schedule for the rest of 2011
"Happy-melon" writes: > Development work done in branches might as well be left in a working > copy for all the attention it gets I'm not about to say that development branches get the love and attention of trunk — just look at the mess we have with with iwtransclusion branch. Here we are about 9 months later and the GSOC code is just now being merged. (https://bugzilla.wikimedia.org/28673) We are learning from that mistake, though. I think Sumanah is directing current GSOC students to work on trunk instead of a branch. > BZ patches doubly so. I'm watching the front of the bug stream and, usually, when someone finds bug and provides a patch, I apply it right away. If someone does that enough (/me waves at Paul Copperman) then we try to get them commit access. Of course, things aren't perfect, but hopefully they are getting better. Mark. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
I think the idea that we might break the existing PHP "parser" out into a library for general use is rather silly. The "parser" is not a parser, it's a macro expander with a pile of regular-expressions used to convert short-hand HTML into actual HTML. The code that it outputs is highly dependent on the state of the wiki's configuration and database content at the moment of "parsing". It also is useless to anyone wanting to do anything other than render a page into HTML, because the output is completely opaque as to where any of it was derived. Dividing the "parser" off into a library would require an substantial amount of MediaWiki code to be ported too just to get it working. On it's own, it would be essentially useless. So, it's probably not an issue what license this hypothetical code would be released under. - Trevor On Tue, May 3, 2011 at 1:25 PM, David Gerard wrote: > On 3 May 2011 21:15, Domas Mituzas wrote: > > >> Thoughts? Also, for re-licensing, what level of approval do we need? > >> All authors of the parser, or the current people in an svn blame? > > > Current people are doing 'derivative work' on previous authors work. I > think all are needed. Pain oh pain. > > > This is the other reason to reduce it to mathematics, which can then > be freely reimplemented. > > > - d. > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On 3 May 2011 21:15, Domas Mituzas wrote: >> Thoughts? Also, for re-licensing, what level of approval do we need? >> All authors of the parser, or the current people in an svn blame? > Current people are doing 'derivative work' on previous authors work. I think > all are needed. Pain oh pain. This is the other reason to reduce it to mathematics, which can then be freely reimplemented. - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Zend performance (Was: WYSIWYG and parser plans)
Hi, I'm not sure what you are profiling, but when repeatingly requesting a preview of an article containing 20 bytes of data consisting of the pattern "a a a a a a " I got the below results. (The php parser doesn't seem to depend on perl regexps.) CPU: CPU with timer interrupt, speed 0 MHz (estimated) Profiling through timer interrupt samples %app name symbol name 994 23.4933 libpcre.so.3.12.1/lib/libpcre.so.3.12.1 545 12.8811 libphp5.so zendparse 369 8.7213 libphp5.so lex_scan 256 6.0506 libc-2.11.2.so memcpy 137 3.2380 libphp5.so zend_hash_find 135 3.1907 libphp5.so _zend_mm_alloc_canary_int 105 2.4817 libphp5.so __i686.get_pc_thunk.bx 902.1272 libphp5.so _zend_mm_free_canary_int 671.5835 libphp5.so zif_strtr 591.3945 libphp5.so zend_mm_add_to_free_list 481.1345 libphp5.so zend_mm_remove_from_free_list /Andreas 2011-05-03 14:40, Domas Mituzas skrev: > some oprofile data showsthat pcre is few percent of execution time - and > there's really lots of Zend internals that stand in the way - memory > management (HPHP implements it as C++ object allocations via jemalloc), > symbol resolutions (native calls in C++), etc. > > Domas > > samples %image name app name symbol > name > 4924009.6648 libphp5.so libphp5.so > _zend_mm_alloc_int > 4515738.8634 libc-2.7.so libc-2.7.so (no > symbols) > 3478126.8268 libphp5.so libphp5.so > zend_hash_find > 3456656.7847 no-vmlinux no-vmlinux (no > symbols) > 3305136.4873 libphp5.so libphp5.so > _zend_mm_free_int > 2257554.4311 libpcre.so.3.12.1libpcre.so.3.12.1(no > symbols) > 1599253.1390 libphp5.so libphp5.so > zend_do_fcall_common_helper_SPEC > 1377092.7029 libphp5.so libphp5.so > _zval_ptr_dtor > 1272332.4973 libxml2.so.2.6.31libxml2.so.2.6.31(no > symbols) > 1112492.1836 libphp5.so libphp5.so > zend_hash_quick_find > 93994 1.8449 libphp5.so libphp5.so > _zend_hash_quick_add_or_update > 84693 1.6623 libphp5.so libphp5.so > zend_assign_to_variable > 84256 1.6538 fss.so fss.so (no > symbols) > 56474 1.1085 libphp5.so libphp5.so execute > 49959 0.9806 libphp5.so libphp5.so > zend_hash_destroy > 48450 0.9510 libz.so.1.2.3.3 libz.so.1.2.3.3 (no > symbols) > 46967 0.9219 libphp5.so libphp5.so > ZEND_JMPZ_SPEC_TMP_HANDLER > 46523 0.9131 libphp5.so libphp5.so > _zend_hash_add_or_update > 45747 0.8979 libphp5.so libphp5.so > zend_str_tolower_copy > 39154 0.7685 libphp5.so libphp5.so > zend_fetch_dimension_address > 35356 0.6940 libphp5.so libphp5.so > ZEND_RECV_SPEC_HANDLER > 33381 0.6552 libphp5.so libphp5.so > compare_function > 32660 0.6410 libphp5.so libphp5.so > _zend_hash_index_update_or_next_insert > 31815 0.6245 libphp5.so libphp5.so > zend_parse_va_args > 31689 0.6220 libphp5.so libphp5.so > ZEND_SEND_VAR_SPEC_CV_HANDLER > 31554 0.6193 libphp5.so libphp5.so _emalloc > 30404 0.5968 libphp5.so libphp5.so > _get_zval_ptr_var > 29812 0.5851 libphp5.so libphp5.so > ZEND_ASSIGN_REF_SPEC_CV_VAR_HANDLER > 28092 0.5514 libphp5.so libphp5.so > ZEND_DO_FCALL_SPEC_CONST_HANDLER > 27760 0.5449 libphp5.so libphp5.so > zend_hash_clean > 27589 0.5415 libphp5.so libphp5.so > zend_fetch_var_address_helper_SPEC_CONST > 26731 0.5247 libphp5.so libphp5.so > _zval_dtor_func > 24732 0.4854 libphp5.so libphp5.so > ZEND_ASSIGN_SPEC_CV_VAR_HANDLER > 24732 0.4854 libphp5.so libphp5.so > ZEND_RECV_INIT_SPEC_CONST_HANDLER > 22587 0.4433 libphp5.so libphp5.so > zend_send_by_var_helper_SPEC_CV > 22176 0.4353 libphp5.so libphp5.so _efree > 21911 0.4301 libphp5.so libphp5.so .plt > 21102 0.4142 lib
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
> > Thoughts? Also, for re-licensing, what level of approval do we need? > All authors of the parser, or the current people in an svn blame? Current people are doing 'derivative work' on previous authors work. I think all are needed. Pain oh pain. Domas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87351]: New comment added
User "Reedy" posted a comment on MediaWiki.r87351. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87351#c16521 Commit summary: Only one extension was using getThumbnail Comment: You're mixing tabs and spaces ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87349]: New comment added
User "^demon" posted a comment on MediaWiki.r87349. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87349#c16520 Commit summary: Only one extension was using getThumbnail Comment: Callers were fixed in r87351. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
> /me hopes the OT messages end with this and next one is on topic. > So, getting back on topic, if we are seriously considering telling people to use a hphp compiled version of the parser as a library, we should re-license it in a more compatible license. LGPL is good, BSD is likely better. Thoughts? Also, for re-licensing, what level of approval do we need? All authors of the parser, or the current people in an svn blame? - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87349]: New comment added
User "^demon" posted a comment on MediaWiki.r87349. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87349#c16519 Commit summary: Only one extension was using getThumbnail Comment: This has only been throwing warnings since November (although it's been deprecated since r21411). I'd leave it in for one last release and then remove it. Plus the last callers in /trunk/extensions should be fixed before outright removal. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
On Wed, May 4, 2011 at 1:35 AM, Peter Youngmeister wrote: > If you guys still think about ideas as things that can be stolen, I'm pretty sure that was meant as a joke. Oh wait, what if *this* was also meant as a joke and I didn't get it? :| /me hopes the OT messages end with this and next one is on topic. -- Yuvi Panda T http://yuvi.in/blog ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 05/03/2011 08:28 PM, Neil Harris wrote: > On 03/05/11 19:44, MZMcBride wrote: ... >> The point is that the wikitext and its parsing should be completely separate >> from MediaWiki/PHP/HipHop/Zend. >> >> I think some of the bigger picture is getting lost here. Wikimedia produces >> XML dumps that contain wikitext. For most people, this is the only way to >> obtain and reuse large amounts of content from Wikimedia wikis (especially >> as the HTML dumps haven't been re-created since 2008). There needs to be a >> way for others to be able to very easily deal with this content. >> >> Many people have suggested (with good reason) that this means that wikitext >> parsing needs to be reproducible in other programming languages. While >> HipHop may be the best thing since sliced bread, I've yet to see anyone put >> forward a compelling reason that the current state of affairs is acceptable. >> Saying "well, it'll soon be much faster for MediaWiki to parse" doesn't >> overcome the legitimate issues that re-users have (such as programming in a >> language other than PHP, banish the thought). >> >> For me, the idea that all that's needed is a faster parser in PHP is a >> complete non-starter. >> >> MZMcBride >> > > I agree completely. > > I think it cannot be emphasized enough that what's valuable about > Wikipedia and other similar wikis is the hard-won _content_, not the > software used to write and display it at any given, which is merely a > means to that end. > > Fashions in programming languages and data formats come and go, but the > person-centuries of writing effort already embodied in Mediawiki's > wikitext format needs to have a much longer lifespan: having a > well-defined syntax for its current wikitext format will allow the > content itself to continue to be maintained for the long term, beyond > the restrictions of its current software or encoding format. > > -- Neil +1 to both MZMcBride and Neil. So relieved to see things put so eloquently. Dirk -- Website: http://dirkriehle.com - Twitter: @dirkriehle Ph (DE): +49-157-8153-4150 - Ph (US): +1-650-450-8550 ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87350]: Revision status changed
User "Krinkle" changed the status of MediaWiki.r87350. Old Status: new New Status: ok Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87350#c0 Commit summary: Localisation updates for ToolserverI18N messages from translatewiki.net (2011-05-03 19:57:00 UTC) ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
If you guys still think about ideas as things that can be stolen, perhaps you should check out the open source movement. Here's a good reference: http://en.wikipedia.org/wiki/Open_source On Tue, May 3, 2011 at 11:21 AM, Ryan Lane wrote: >> This is how WMF staff treats volunteers: >> >> [21:17:23] domas: and now I took your BSD idea, and didn't give >> you credit >> [21:17:38] * Ryan_Lane wins >> [21:17:51] FLAWLESS VICTORY >> [21:17:55] except for the IRC logs >> > > You are evil Domas. For those interested, check the logs higher up, > where I discuss licensing way before Domas forked the email thread. He > steals my licensing idea, I steal his specific license. :D > > - Ryan > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87310]: New comment added
User "Krinkle" posted a comment on MediaWiki.r87310. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87310#c16518 Commit summary: Adding sql to create table for article feedback stats relating to high/low article ratings; Adding maintenance script to periodically populate high/low article feedback stats table Comment: The [http://php.net/manual/en/class.invalidargumentexception.php InvalidArgumentException class] is in PHP core. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Bugzilla Weekly Report
If it's not too much work, I'd definitly like to see this once as a trial. Especially the inclusion of links and summaries is useful imho. -- Krinkle On 3 May 2011, at 18:43, wrote: > Fwiw a MeeGo contributor has created a script to retrieve weekly > information from http://bugs.meego.com and make these nice and > useful reports: > > http://www.octofish.net/meegobugjar/platform/ > > Let me know if you are interested in an equivalent for > bugzilla.wikimedia.org > > -- > Quim > > On Monday, May 02, 2011 5:35 PM, Tomasz Finc wrote: >> Indeed .. i remember writing this report for brion years ago. >> brings back >> memories. >> >> --tomasz > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Release schedule for the rest of 2011 (was: Status of 1.17 & 1.18)
I find fine planning to have a "normal", small release. But if the code development lead to eg. a parser rewrite, that's fine too. It can get in, or reverted for the branch if too close to the brach point. As far as it gets reviewed in time, it shouldn't be a problem. (We are going to get everything timely reviewed™ this time, right? ;) However, if it gets unreviewed three months, with furhter changes dependant of it, and is only looked at two weeks before the branching date (and obviously finding issues), then such rewrite becomes a big problem. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87041]: New comment added
User "Krinkle" posted a comment on MediaWiki.r87041. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87041#c16517 Commit summary: Fixes for IE7 (alignment of checkboxes due to double-margin bug on floated elements. The popular display:inline hack doesn't fix this. Removed the margin for now, was collapsed/invisible in most browsers anyway. Covered by the margins on the parent elements. Comment: This was reported as bug 28721 by the way. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87310]: New comment added
User "Awjrichards" posted a comment on MediaWiki.r87310. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87310#c16516 Commit summary: Adding sql to create table for article feedback stats relating to high/low article ratings; Adding maintenance script to periodically populate high/low article feedback stats table Comment: done. thanks reedy! i've been forgetting to do this for a long time... ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
[MediaWiki-CodeReview] [MediaWiki r87339]: New comment added
User "Bawolff" posted a comment on MediaWiki.r87339. Full URL: http://www.mediawiki.org/wiki/Special:Code/MediaWiki/87339#c16515 Commit summary: Fixes bug #28694: Set maxlength of revisiondelete comment to 100. Comment: Presumably this limit is due to limitation on db end? We should probably use the byteCounting js so that it works properly with multi-byte utf-8 characters properly (since the db limit would be a byte limit not a character limit), like is done in resources/mediawiki.special/mediawiki.special.movePage.js. ___ MediaWiki-CodeReview mailing list mediawiki-coderev...@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] WYSIWYG and parser plans (was What is wrong with Wikia's WYSIWYG?)
On 03/05/11 19:44, MZMcBride wrote: > Chad wrote: >> On Tue, May 3, 2011 at 2:15 PM, MZMcBride wrote: >>> I realize you have a dry wit, but I imagine this joke was lost on nearly >>> everyone. You're not really suggesting that everyone who wants to parse >>> MediaWiki wikitext compile and run HipHop PHP in order to do so. >> And how is using the parser with HipHop going to be any more >> difficult than using it with Zend? > The point is that the wikitext and its parsing should be completely separate > from MediaWiki/PHP/HipHop/Zend. > > I think some of the bigger picture is getting lost here. Wikimedia produces > XML dumps that contain wikitext. For most people, this is the only way to > obtain and reuse large amounts of content from Wikimedia wikis (especially > as the HTML dumps haven't been re-created since 2008). There needs to be a > way for others to be able to very easily deal with this content. > > Many people have suggested (with good reason) that this means that wikitext > parsing needs to be reproducible in other programming languages. While > HipHop may be the best thing since sliced bread, I've yet to see anyone put > forward a compelling reason that the current state of affairs is acceptable. > Saying "well, it'll soon be much faster for MediaWiki to parse" doesn't > overcome the legitimate issues that re-users have (such as programming in a > language other than PHP, banish the thought). > > For me, the idea that all that's needed is a faster parser in PHP is a > complete non-starter. > > MZMcBride > I agree completely. I think it cannot be emphasized enough that what's valuable about Wikipedia and other similar wikis is the hard-won _content_, not the software used to write and display it at any given, which is merely a means to that end. Fashions in programming languages and data formats come and go, but the person-centuries of writing effort already embodied in Mediawiki's wikitext format needs to have a much longer lifespan: having a well-defined syntax for its current wikitext format will allow the content itself to continue to be maintained for the long term, beyond the restrictions of its current software or encoding format. -- Neil ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[MediaWiki-CodeReview] [MediaWiki r87310]: New comment added, and revision status changed
User "Catrope" changed the status of MediaWiki.r87310.
Old Status: new
New Status: fixme
User "Catrope" also posted a comment on MediaWiki.r87310.
Full URL:
https://secure.wikimedia.org/wikipedia/mediawiki/wiki/Special:Code/MediaWiki/87310#c16514
Commit summary:
Adding sql to create table for article feedback stats relating to high/low
article ratings; Adding maintenance script to periodically populate high/low
article feedback stats table
Comment:
+ public function formatTimestamp( $unix_time ) {
+ if ( !is_numeric( $unix_time )) {
+ throw new InvalidArgumentException( 'Timestamp must be
numeric.' );
+ }
+ return date( 'Ymdhis', $unix_time );
+ }
We have this built-in as $dbw->timestamp() . Remember that
different DBMSes have different timestamp formats. When converting timestamps
for non-DB purposes, there's wfTimestamp() . Also, there is no
InvalidArgumentException class that I can find.
+ 'afshl_avg_ratings' => json_encode( $data[
'avg_ratings' ] ),
json_encode() is not guaranteed to be available or non-broken, use
FormatJson::encode()
+ array( array( 'afshl_page_id',
'afshl_avg_overall', 'afshl_avg_ratings', 'afshl_ts' )),
If this set of fields is supposed to form a unique combination, you have to add
a UNIQUE index on it, or this replace call will work in Postgres but not in
MySQL. Generally, the table doesn't have any indices other than the primary key.
+ array(
+ 'aa_page_id',
+ 'avg(aa_rating_value) as avg_rating'
+ ),
+ 'aa_timestamp > ' . $this->getTimestamp(),
+ __METHOD__,
+ array(
+ 'GROUP BY' => 'aa_page_id',
+ 'ORDER BY' => 'avg_rating DESC',
+ 'LIMIT' => '50',
+ )
This query sounds painful. There is no index on aa_timestamp, for one thing.
I'm not sure what the index would have to be exactly in this case because the
GROUP BY and ORDER BY are different, I'd have to look that up.
+ 'aa_timestamp > ' . $this->getTimestamp() . ' &&
aa_page_id IN (' . $page_id . ')',
&& instead of AND in SQL? Does that even work? Also,
you don't have to build your own IN(...) list, just use
array( "aa_timestamp >= $blah", 'aa_page_id' => $arrayOfPageIDs )
___
MediaWiki-CodeReview mailing list
mediawiki-coderev...@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-codereview
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
> This is how WMF staff treats volunteers: > > [21:17:23] domas: and now I took your BSD idea, and didn't give > you credit > [21:17:38] * Ryan_Lane wins > [21:17:51] FLAWLESS VICTORY > [21:17:55] except for the IRC logs > You are evil Domas. For those interested, check the logs higher up, where I discuss licensing way before Domas forked the email thread. He steals my licensing idea, I steal his specific license. :D - Ryan ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Licensing (Was: WYSIWYG and parser plans)
Hi! > I was just talking about this in IRC :). We could re-license the > parser to be LGPL or BSD so that other implementations can use our > parser more freely. This is how WMF staff treats volunteers: [21:17:23] domas: and now I took your BSD idea, and didn't give you credit [21:17:38] * Ryan_Lane wins [21:17:51] FLAWLESS VICTORY [21:17:55] except for the IRC logs Domas ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
