Re: [Wikitech-l] On templates and programming languages
On Wed, Jul 1, 2009 at 1:42 AM, Dmitriy Sintsovques...@rambler.ru wrote: XSLT itself is a way too much locked down - even simple things like substrings manipulation and loops aren't so easy to perform. Well, maybe I am too stupid for XSLT but from my experience bringing tag syntax in programming language make the code poorly readable and bloated. I've used XSLT for just one of my projects. Juniper Networks (my day job) uses XSLT as the primary scripting language on their routing devices, and chose to do so primarily because of sandboxing and the ease of XML tree manipulation with xpath (JunOS configuration has a complete and comprehensive XML representation). To facilitate that usage we defined an alternative syntax for XSLT called SLAX (http://code.google.com/p/libslax/), though it hasn't seen widespread adoption outside of Juniper yet. (Slax can be mechanically converted to XSLT and vice versa) SLAX pretty much resolves your readability concern. Although there are the conceptual barriers for people coming from procedural languages to any strongly functional programming language still remain. You don't loop in XSLT, you recurse or iterate over a structure (i.e. map/reduce). I've grown rather fond of XSLT but wouldn't personally recommend it for this application. It lacks the high speed bytecoded execution environments available for other languages, snf I don't see many scripts on the site doing extensive document tree manipulation (it's hard for me to express how awesome xpath is at that)... and I would also guess that there are probably more adept mediawiki template language coders today than there are people who are really fluent in XSLT. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] secure slower and slower
I've been using secure for login for months now, and at first it seemed pretty good, other than the inability to switch sites easily. (And always editing links from secure.wikimedia.org/.../w to en.wikipedia.org/w, but I've gotten used to doing that extra bit by hand.) Anyway, it's just been a dog lately. During EDT daylight hours, it often gives an error not able to access page, especially saving. So, I've reverted to the old practice from the dog days of 2005-06, and mostly edit in very off hours. Yet it slowed down drastically again! The logs show a problem some hours ago. But nothing recent. Are there too many secure users? I've found no way to see the status. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
Haven't read the entire thread yet, so hopefully nobody has said this: Perl, write-once, poor choice for uncontrolled environment. Lisp, at least the computer science type will know. Haven't used it myself since early '80s. Lua, don't know whether it's improved in the past few years, but freeciv had serious problems with migrating to 5.1. Personally, I've given up on it, but my 14 y-o nephew seems to like it for various game modification. Javascript, OMG don't go there. Everybody seems to be going the python direction lately, but I've only minimal experience with it, so cannot make a recommendation. I'd worry less about providing extensive functionality (we certainly don't have much now, so anything more would be gravy), but rather ease of integration, scalability, and security. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
On Wed, Jul 1, 2009 at 3:50 AM, William Allen Simpsonwilliam.allen.simp...@gmail.com wrote: Javascript, OMG don't go there. Don't be so quick to dismiss Javscript. If we were making a scorecard it would likely meet most of the checkboxes: * Available of reliable battle tested sandboxes (and probably the only option discussed other than x-in-JVM meeting this criteria) * Availability of fast execution engines * Widely known by the existing technical userbase (JS beats the other options hands down here) * Already used by many Mediawiki developers * Doesn't inflate the number of languages used in the operation of the site * Possibility of reuse between server-executed and client-executed (Only JS of the named options meets this criteria) * Can easily write clear and readable code * Modern high level language features (dynamic arrays, hash tables, etc) There may exist great reasons why another language is a better choice, but JS is far from the first thing that should be eliminated. Python is a fine language but it fails all the criteria I listed above except the last two. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
Javascript might have gotten a bad name in the past because of 14-year olds who used it to display 'Welcome to my website!' alerts on their Geocities homepage, but it's really unfair. Javascript is a very flexible and dynamic language that can be written very elegantly. I urge everyone who still think Javascript is a toy language to read Douglas Crockford's excellent article: http://javascript.crockford.com/javascript.html -- Hay On Wed, Jul 1, 2009 at 10:35 AM, Gregory Maxwellgmaxw...@gmail.com wrote: On Wed, Jul 1, 2009 at 3:50 AM, William Allen Simpsonwilliam.allen.simp...@gmail.com wrote: Javascript, OMG don't go there. Don't be so quick to dismiss Javscript. If we were making a scorecard it would likely meet most of the checkboxes: * Available of reliable battle tested sandboxes (and probably the only option discussed other than x-in-JVM meeting this criteria) * Availability of fast execution engines * Widely known by the existing technical userbase (JS beats the other options hands down here) * Already used by many Mediawiki developers * Doesn't inflate the number of languages used in the operation of the site * Possibility of reuse between server-executed and client-executed (Only JS of the named options meets this criteria) * Can easily write clear and readable code * Modern high level language features (dynamic arrays, hash tables, etc) There may exist great reasons why another language is a better choice, but JS is far from the first thing that should be eliminated. Python is a fine language but it fails all the criteria I listed above except the last two. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
-Original Message- From: wikitech-l-boun...@lists.wikimedia.org [mailto:wikitech-l-boun...@lists.wikimedia.org] On Behalf Of Brion Vibber Sent: 30 June 2009 17:17 To: Wikimedia developers Subject: [Wikitech-l] On templates and programming languages As many folks have noted, our current templating system works ok for simple things, but doesn't scale well -- even moderately complex conditionals or text-munging will quickly turn your template source into what appears to be line noise. And we all thought Perl was bad! ;) There's been talk of Lua as an embedded templating language for a while, and there's even an extension implementation. One advantage of Lua over other languages is that its implementation is optimized for use as an embedded language, and it looks kind of pretty. An _inherent_ disadvantage is that it's a fairly rarely-used language, so still requires special learning on potential template programmers' part. An _implementation_ disadvantage is that it currently is dependent on an external Lua binary installation -- something that probably won't be present on third-party installs, meaning Lua templates couldn't be easily copied to non-Wikimedia wikis. There are perhaps three primary alternative contenders that don't involve making up our own scripting language (something I'd dearly like to avoid): * PHP Advantage: Lots of webbish people have some experience with PHP or can easily find references. Advantage: we're pretty much guaranteed to have a PHP interpreter available. :) Disadvantage: PHP is difficult to lock down for secure execution. * JavaScript Advantage: Even more folks have been exposed to JavaScript programming, including Wikipedia power-users. Disadvantage: Server-side interpreter not guaranteed to be present. Like Lua, would either restrict our portability or would require an interpreter reimplementation. :P * Python Advantage: A Python interpreter will be present on most web servers, though not necessarily all. (Windows-based servers especially.) Wash: Python is probably better known than Lua, but not as well as PHP or JS. Disadvantage: Like PHP, Python is difficult to lock down securely. Any thoughts? Does anybody happen to have a PHP implementation of a Lua or JavaScript interpreter? ;) Would you want the interpreter to translate the template into PHP array of opcodes first, so could dump that into APC/MemCache? Jared ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
I'm glad to see I'm not alone. JavaScript can indeed invoke bad memories of fragile scripts running in IE5 which are long and awkward due to limitations in browser technology at the time. However, anyone who has used a modern library like jQuery on a support browser will tell you it's very powerful and intuitive while being simple, straightforward and actually fun. Any language capable of supporting this experience is worth seriously considering as an option for us. - Trevor Sent from my iPod On Jul 1, 2009, at 1:44 AM, Hay (Husky) hus...@gmail.com wrote: Javascript might have gotten a bad name in the past because of 14-year olds who used it to display 'Welcome to my website!' alerts on their Geocities homepage, but it's really unfair. Javascript is a very flexible and dynamic language that can be written very elegantly. I urge everyone who still think Javascript is a toy language to read Douglas Crockford's excellent article: http://javascript.crockford.com/javascript.html -- Hay On Wed, Jul 1, 2009 at 10:35 AM, Gregory Maxwellgmaxw...@gmail.com wrote: On Wed, Jul 1, 2009 at 3:50 AM, William Allen Simpsonwilliam.allen.simp...@gmail.com wrote: Javascript, OMG don't go there. Don't be so quick to dismiss Javscript. If we were making a scorecard it would likely meet most of the checkboxes: * Available of reliable battle tested sandboxes (and probably the only option discussed other than x-in-JVM meeting this criteria) * Availability of fast execution engines * Widely known by the existing technical userbase (JS beats the other options hands down here) * Already used by many Mediawiki developers * Doesn't inflate the number of languages used in the operation of the site * Possibility of reuse between server-executed and client-executed (Only JS of the named options meets this criteria) * Can easily write clear and readable code * Modern high level language features (dynamic arrays, hash tables, etc) There may exist great reasons why another language is a better choice, but JS is far from the first thing that should be eliminated. Python is a fine language but it fails all the criteria I listed above except the last two. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
William Allen Simpson wrote: I run Firefox with JS off by default for all wikimedia sites, because of serious problems in the not so recent past! s/recent/distant/ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
I run Firefox with JS off by default for all wikimedia sites, because of serious problems in the not so recent past! s/recent/distant/ Hooray JavaScript FUD! ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] secure slower and slower
William Allen Simpson wrote: So, I've reverted to the old practice from the dog days of 2005-06, and mostly edit in very off hours. Yet it slowed down drastically again! Here's my test log, edits queued and ready to go, demonstrating roughly how long they take to come back and display: ;off hours last night # 2009-07-01T06:54:45 # 2009-07-01T06:55:59 1 minute 14 seconds ;peak time today # 2009-07-01T17:05:24 # 2009-07-01T17:06:17 53 seconds # 2009-07-01T17:06:53 36 seconds # 2009-07-01T17:08:00 1 minute 7 seconds # 2009-07-01T17:08:40 40 seconds # 2009-07-01T17:09:45 1 minute 5 seconds # 2009-07-01T17:11:49 2 minutes 4 seconds # 2009-07-01T17:12:44 55 seconds # 2009-07-01T17:13:49 1 minute 5 seconds # 2009-07-01T17:15:00 1 minute 11 seconds # 2009-07-01T17:16:10 1 minute 10 seconds In short, just as slow today at peak as off peak. Something wrong! The logs show a problem some hours ago. But nothing recent. Are there too many secure users? I've found no way to see the status. https://wikitech.leuksman.com/view/Server_admin_log linked from http://ganglia.wikimedia.org/ no longer works, bad certificate. http://wikitech.wikimedia.org/view/Server_admin_log Last entry at 16:13. http://www.thewritingpot.com/wikistatus/ Fast/Fast, no changes since 19 hours ago ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
On Wed, Jul 1, 2009 at 11:21 AM, William Allen Simpsonwilliam.allen.simp...@gmail.com wrote: * Doesn't inflate the number of languages used in the operation of the site This is the important checkbox, as far as integration with the project (my first criterion), but is the server side code already running JavaScript? For serving pages? No but mediawiki and the sites are already chock-full of client side code in JS. You basically can't do advanced development for MediaWiki or the wikimedia sites without a degree of familiarity with Javascript due to client compatibility considerations. My general rule: coming over the network, presume it's bad data. In this case were not talking about the language mediawiki is written in, we're talking about a language used for server-side content automation (templates). In that case we'd be assuming the inputs are toxic just like in the client side case, since everything, including the code itself came in over the network. I'll concede that there likely wouldn't be much code reuse, but I'd attribute that more to the starkly different purpose and the fact that the server version would have a different API (no DOM, but instead functions for pulling data out of mediawiki). And we have far too many examples of existing JS already being used in horrid templates, being promulgated in important areas such as large categories, that don't seem to work consistently, and don't work at all with JavaScript turned off. I run Firefox with JS off by default for all wikimedia sites, because of serious problems in the not so recent past! Fortunately this is a non-issue here: Better server side scripting enhances the sites ability to operate without requiring scripting on the client. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
Hay (Husky) wrote:
I'm sorry that you seem to have such bad experiences with JavaScript.
Still, i don't think your comments are really valid in today's world.
You mean like the {{hidden}} template series? How long that took to
finally work?
Worse, folks trying to use the classes directly, resulting in the contents
being centered, with the bullets and numbering removed:
div class=NavFrame collapsed
div class=NavHeadCategories/div
div class=NavContent
https://secure.wikimedia.org/wikipedia/en/w/index.php?title=Wikipedia:Categories_for_discussion/Log/2009_June_6diff=294783332oldid=294782497
or
http://en.wikipedia.org/w/index.php?title=Wikipedia:Categories_for_discussion/Log/2009_June_6diff=294783332oldid=294782497
Believe me, user edits relying on JS, even where the JS isn't directly
accessible, are not really ready for prime time today.
Take a look at 'web 2.0-style' applications, such as Gmail or Google
Maps. Stuff like that would simply be impossible in a web browser
without depending on proprietary technology such as Flash.
Sure, and do you know how many months it took to get that to work, or
how many folks from the application security group to review?
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
On Wed, 01 Jul 2009 09:42:31 +0400, Dmitriy Sintsov wrote:
XSLT itself is a way too much locked down - even simple things like
substrings manipulation and loops aren't so easy to perform. Well, maybe
I am too stupid for XSLT but from my experience bringing tag syntax in
programming language make the code poorly readable and bloated. I've
used XSLT for just one of my projects.
I'd assume we want locked down. Loops would be hard in any locked-down
environment; I don't recall seeing any recommendation in this thread on
how that wold be done. Recursion is much simpler, just track the depth,
and throw an exception if it goes to deep; emacs lisp already uses this
mechanism.
Some of those things may not be as easy as other lanugages, but the string
functions that this thread was started over are built into XPath 2.0, so
it would solve the problem at hand.
Deeply nested braces of lisp remind me of current MediaWiki parser.
Superficially, sure; but IMHO the real problem with the current parser is
the ambiguity, that when you see a construct begin like {something...
you need to keep reading before you can parse it. With lisp, it's trivial
to parse, so we could do our own parsing if needed.
Lua was highly valued here at computer lab, also Ocaml (not sure of
proper spelling).
Dmitriy
It seems like there are benefits there, but it's less clear how to
implement that sufficiently locked down, and how that would interface with
the rest of the parser, for callbacks, magic words, etc.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] On templates and programming languages
On Tue, 30 Jun 2009 22:53:36 +0100, Thomas Dalton wrote:
2009/6/30 Steve Sanbeg ssan...@ask.com:
On Tue, 30 Jun 2009 21:38:07 +0100, Thomas Dalton wrote:
2009/6/30 Michael Daly michael.d...@kayakwiki.org:
How does that work with anonymous variables? Are all $[NUMBER] style
names count as auto-declared?
They're not anonymous, they're just named sequentially.
They are anonymous when you call the template, though. The names are
determined by the order in the call rather than written explicitly.
They do need to be considered separately.
Anonymous would mean they don't have names, which isn't the case. They
are named, but those names may, or may not, be implicit. Currently, they
aren't handled separately; the parser names any unnamed arguments prior to
calling the template, which has no way of knowing how they were named; to
the template, they're all just named arguments {{t|a|b}} is the same as
{{t|2=b|1=a}} or even {{t|2=a|b}}.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] $300K grant for Wikimedia Commons
All, I'm very happy to announce that the Ford Foundation has awarded a $300,000 grant to the Wikimedia Foundation to improve our interfaces and workflows for multimedia uploading. Press release here: http://wikimediafoundation.org/wiki/Press_releases/Wikimedia_Ford_Foundation_Grant_July_2009 For the first time we're also sharing a full grant proposal, with permission of the Ford Foundation. You can find it here: http://upload.wikimedia.org/wikipedia/foundation/f/f9/WMF_Ford_Multimedia_Participation_Project.pdf It should give you a good idea about what we can do within the scope of this project. As a brief recap, Michael Dale has already done some good work on external repository searches and transfers, and integration of uploading into the editing UI, so we're hoping to build on top of this to really get the workflow for licensing/upload/review/embedding of media files nailed. We've also been having initial discussions with some of the Wikimedia chapters about possible models for working together on the execution of this. For example, we want to make sure that we can facilitate fruitful face-to-face meetings with Commons practitioners, and there is plenty of technical work to be done that can be decentralized and shared. Exciting projects like Wikimedia Germany's investment in multilingual search are already underway, so hopefully over the next year, we'll see lots of useful activity culminating in genuine improvements for Commons and beyond. Big thanks to Sara Crouse and Naoko Komura for their work on this grant proposal, and of course to the Ford Foundation for funding it. :-) Wikimedia Commons deserves to grow to many more millions of free educational media files, and hopefully this strategic investment will help us to get there. All best, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] $300K grant for Wikimedia Commons
On Thu, Jul 2, 2009 at 11:30 AM, Erik Moellere...@wikimedia.org wrote: http://upload.wikimedia.org/wikipedia/foundation/f/f9/WMF_Ford_Multimedia_Participation_Project.pdf The objective of this project is to increase participation in and contributions to Wikimedia Commons by implementing a 13month software development, usability testing and documentation project to improve the interface for uploading multimedia files to Wikimedia Commons. The deliverables should include the following key improvements: •an integrated upload tool that can be accessed directly from the editing window; •Wikipedia integration of Wikimedia Commons as a repository to store freely licensed media; •an intelligent workflow for fair use media that are not permissible on Wikimedia Commons; •an upload form process that emphasizes common defaults above less frequent use cases; •separated instructions and tutorials for conveying key policy information and background on copyright law and licensing. And thank god for that! :) I can't wait. Steve ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] $300K grant for Wikimedia Commons
This is excellent news. For those who haven't seen it in action, Michael Dale's work on the upload workflow is looking awesome. I'm sure this grant will make it even better. -Chad On Jul 1, 2009 11:52 PM, Steve Bennett stevag...@gmail.com wrote: On Thu, Jul 2, 2009 at 11:30 AM, Erik Moellere...@wikimedia.org wrote: http://upload.wikimedia.org... The objective of this project is to increase participation in and contributions to Wikimedia Commons by implementing a 13month software development, usability testing and documentation project to improve the interface for uploading multimedia files to Wikimedia Commons. The deliverables should include the following key improvements: •an integrated upload tool that can be accessed directly from the editing window; •Wikipedia integration of Wikimedia Commons as a repository to store freely licensed media; •an intelligent workflow for fair use media that are not permissible on Wikimedia Commons; •an upload form process that emphasizes common defaults above less frequent use cases; •separated instructions and tutorials for conveying key policy information and background on copyright law and licensing. And thank god for that! :) I can't wait. Steve ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
