[Wikitech-l] Reminder: Bug triage today at 17:00 UTC at #wikimedia-dev

[matanya]

Hi,

please see

https://www.mediawiki.org/wiki/Bug_management/Triage

and the etherpad ready with bugs:

http://etherpad.wikimedia.org/BugTriage-2012-09

Hope seeing you on IRC!

Matanya



signature.asc
Description: OpenPGP digital signature
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Changes to the API for Wikidata

[John Erling Blad]

Please note that this is a breaking change for bots!

It is decided that the module wbsetitem will change from the present
"short form" in the json structure to a "long form". Exactly how the
long form will be is still a bit open, but it will be closer to the
json output format. The changes also makes it possible to use a
key-less format as the language and site id will be available inside
the long form.

The following short form will be WRONG in the future
{
  "labels":{
"de":"Foo",
"en":"Bar"
  }
}

This long form will be RIGHT in the future
{
  "labels":{
"de":{"value":"Foo","language":"de"},
"en":{"value":"Bar","language":"en"}
  }
}

And also this will be RIGHT
{
  "labels":[
{"value":"Foo","language":"de"},
{"value":"Bar","language":"en"}
  ]
}

New modules may use a long form if they support json input, but that
for the future.

In some cases it seems necessary to add flags for what to do with
individual fields, but perhaps we can avoid it. (Typically for
adding/removing aliases, but it could also be necessary for other
fields.)

There are also a new "clear" URL argument that will clear the content
of an item and make it possible to rebuild it from scratch. Default
behavior is NOT to clear the content, but to incrementally build on
the existing item.

Also note that use of [no]usekeys in the URL is not supported anymore.

See also
* http://www.mediawiki.org/wiki/Extension:Wikibase/API#New_long_format

/jeblad

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Moving MediaWikiWidgets.org to Wikimedia

[Dmitriy Sintsov]

4 Сентябрь 2012 г. 21:40:20 пользователь John Du Hart (compwhi...@gmail.com) 
написал:


Does MediaWikiWiki really need any more shitty/insecure addons that no
one is going to maintain? I think we have enough already.

Pick out the best of the bunch and nuke the rest.


It is not easy (at least here in Russia) to earn anything via programming 
MediaWiki extensions. I made some extensions, two for local university and 3 or 
4 as freelancer job. They were paid just once. And you have to write them in 
fixed time, that's why many parts of code aren't cleaned / polished up. While 
new versions of MediaWiki introduce new functionality, sometimes 
incompatibility with old extensions. It seems that extension-writing must be 
full-time paid job, however nobody of employers here wants that. They want 
minimal, single-time expenses. I guess that's why there are many abandoned and 
low-quality extensions. Struggling with low income problems is not nice thing, 
especially for the people who have families.
Dmitriy

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] scaled media (thumbs) as *temporary* files, not stored forever

[Roan Kattouw]

On Sun, Sep 2, 2012 at 5:59 PM, Tim Starling  wrote:
> The other reason for the existence of the backend thumbnail store is
> to transport images from the thumbnail scalers to the 404 handler. For
> that purpose, the image only needs to exist in the backend for a few
> seconds. It could be replaced by a better 404 handler, that sends
> thumbnails directly by HTTP. Maybe the Swift one does that already.
>
My understanding is that thumb.php already streamed the thumbnail back
to the 404 handler via HTTP and has done so for at least the past two
years or so.

Roan

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] scaled media (thumbs) as *temporary* files, not stored forever

[Jon Robson]

I just wanted to clarify something... is there any protection in place in
the thumbnail generator to prevent denial of service attacks? For instance
if someone wanted to they could run a script which uploaded photos then
fired off requests for thumbnails of it of size 20px,21px,22px...1024px

I'm guessing the servers wouldn't like that. This is why I'd be keen to
limit the sizes.

May I suggest someone analyses the sizes currently used on wikipedia and we
limit to those as an initial step and then review the less frequently used
ones and standardise on some sizes?
On Sep 5, 2012 9:15 AM, "Roan Kattouw"  wrote:

> On Sun, Sep 2, 2012 at 5:59 PM, Tim Starling 
> wrote:
> > The other reason for the existence of the backend thumbnail store is
> > to transport images from the thumbnail scalers to the 404 handler. For
> > that purpose, the image only needs to exist in the backend for a few
> > seconds. It could be replaced by a better 404 handler, that sends
> > thumbnails directly by HTTP. Maybe the Swift one does that already.
> >
> My understanding is that thumb.php already streamed the thumbnail back
> to the 404 handler via HTTP and has done so for at least the past two
> years or so.
>
> Roan
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] +2 on github (maintaining code quality via an informal review process)

[Jon Robson]

I wanted to raise an issue that the mobile team has recently
discovered during development of the Wiki Loves Monuments mobile app
[1] on github.

Wikimedia has a github account with currently 18 members [2]. Every
single one of these can merge to master on any of the projects.

The mobile team early on decided on a rule that no one can merge their
own code to ensure that we maintain code quality by ensuring that all
commits have been reviewed by at least once person.

This breaks down when people outside the mobile team who don't know
this rule need to commit code to our repository. We've had two
instances recently. One was a bad commit which reverted a change made
by an earlier commit and another could have been improved as it
cleaned up a file that wasn't actually being used and could have just
been removed all together. These things could have been
prevented/improved by our review system on github. Even I have been
caught out by merging my own code in a situation where I thought
someone was a simple change but turned out not to be.

Is this something we need to enforce ourselves (i.e. is there a way to
reject merges where the author is the same) or could we simply agree
and document somewhere that anyone with wikimedia account access on
github should not merge any changes to a project they have made
themselves?

[1] https://github.com/wikimedia/WLMMobile
[2] https://github.com/wikimedia

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 on github (maintaining code quality via an informal review process)

[Yuvi Panda]

I think documentation of how our current model works would be useful -
not just as documentation for people contributing to the mobile
projects but for other projects moving to GitHub as well (I see a few
i18n ones there now!)


-- 
Yuvi Panda T
http://yuvi.in/blog

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] scaled media (thumbs) as *temporary* files, not stored forever

[Asher Feldman]

On Tue, Sep 4, 2012 at 3:11 PM, Platonides  wrote:

> On 03/09/12 02:59, Tim Starling wrote:
> > I'll go for option 4. You can't delete the images from the backend
> > while they are still in Squid, because then they would not be purged
> > when the image is updated or action=purge is requested. In fact, that
> > is one of only two reasons for the existence of the backend thumbnail
> > store on Wikimedia. The thumbnail backend could be replaced by a text
> > file that stores a list of thumbnail filenames which were sent to
> > Squid within a window equivalent to the expiry time sent in the
> > Cache-Control header.
> >
> > The other reason for the existence of the backend thumbnail store is
> > to transport images from the thumbnail scalers to the 404 handler. For
> > that purpose, the image only needs to exist in the backend for a few
> > seconds. It could be replaced by a better 404 handler, that sends
> > thumbnails directly by HTTP. Maybe the Swift one does that already.
> >
> > -- Tim Starling
>
> The second one seems easy to fix. The first one should IMHO be fixed in
> squid/varnish by allowing wildcard purges (ie. PURGE
> /wikipedia/commons/thumb/5/5c/Tim_starling.jpg/* HTTP/1.0)
>

fast.ly  implements group purge for varnish like this via a proxy daemon
that watches backend responses for a "tag" response header (i.e. all
resolutions of Tim_starling.jpg would be tagged that) and builds an
in-memory hash of tags->objects which can be purged on.  I've been told
they'd probably open source the code for us if we want it, and it is
interesting (especially to deal with the fact that we don't purge articles
at all of their possible url's) albeit with its own challenges.  If we
implemented a backend system to track thumbnails that exist for a given
orig, we may be able to remove our dependency on swift container listings
to purge images, paving the way for a second class of thumbnails that are
only cached.

A wiki with such setup could then disable the on-disk storage.
>

I think this is entirely doable, but scaling the imagescalers to support
cache failures at wmf scale would be a waste, except perhaps for
non-standard sizes that aren't widely used.  I like Brion's thoughts on
revamping image handling, and would like to see semi-permanent (in swift)
storage of a standardized set of thumbnail resolutions but we could still
support additional resolutions.  Browser scaling is also at least worth
experimenting with.  Instances where browser scaling would be bad are
likely instances where the image is already subpar if viewed on a high-dpi
/ retina display.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] scaled media (thumbs) as *temporary* files, not stored forever

[Roan Kattouw]

On Wed, Sep 5, 2012 at 12:35 PM, Asher Feldman  wrote:
> Browser scaling is also at least worth
> experimenting with.  Instances where browser scaling would be bad are
> likely instances where the image is already subpar if viewed on a high-dpi
> / retina display.
Other instances where browser scaling is bad are:
* PoS browsers that don't render SVGs (how old are these by now?)
* Even modern browsers have subpar SVG rendering at 1x, PNG looks better
* Some media types are "scaled" in unusual ways (SVGs, but also video
stills, PDF pages, ...)
* Some original images are really friggin' large (20-30 megapixels
sometimes), so at least some downscaling is needed there
* Mobile clients will want to minimize the amount of data transferred

Roan

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] HTML5, it's a coming (again)!

[Sam Reed]

It's been a long time coming (for the nth time..), but we're scheduling a
deployment of HTML5 across the Wikimedia cluster [1]. This is set for Monday
17th September at 18:00-20:00 UTC [2].

 

The intention is to set $wgHtml5 [3] to true everywhere. It's been running
on MediaWiki.org and our 2 test wikis for quite a while, and other sites
like translatewiki.net with no issues.

 

The intention is to leave it enabled unless it causes major problems. If
you're running an application that screen scrapes, shame on you; you've had
enough notice to get it fixed! ;)

 

Now is the time to fix up your scripts and programs (where necessary), tell
your friends!

 

 

Sam

 

 

[1] https://bugzilla.wikimedia.org/show_bug.cgi?id=27478

[2] http://wikitech.wikimedia.org/view/Software_deployments#Week_of_Sept_17

[3] https://www.mediawiki.org/wiki/Manual:$wgHtml5



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] scaled media (thumbs) as *temporary* files, not stored forever

[Brion Vibber]

On Wed, Sep 5, 2012 at 2:00 PM, Roan Kattouw  wrote:

> On Wed, Sep 5, 2012 at 12:35 PM, Asher Feldman 
> wrote:
> > Browser scaling is also at least worth
> > experimenting with.  Instances where browser scaling would be bad are
> > likely instances where the image is already subpar if viewed on a
> high-dpi
> > / retina display.
> Other instances where browser scaling is bad are:
> * PoS browsers that don't render SVGs (how old are these by now?)
>

IE up through 8, and Android stock browser through 2.3. Neither are dead
yet, so we still gotta deal with rasterization for them.

* Even modern browsers have subpar SVG rendering at 1x, PNG looks better
>

Examples? Sounds like bugs need to be filed with some of those browsers. :)


> * Some media types are "scaled" in unusual ways (SVGs, but also video
> stills, PDF pages, ...)
> * Some original images are really friggin' large (20-30 megapixels
> sometimes), so at least some downscaling is needed there
>

You'd absolutely want to do server-side downscaling to the base sizes in
the appropriate file formats -- we wouldn't try to download multi-megapixel
originals just to make a tiny thumbnail, and some formats require
conversion to a format the browser can read.

For an example if we were to standardize on sizes: (we wouldn't use these
actual sizes because they do NOT fit our usage)
* 32px
* 64px
* 128px
* 256px
* 512px
* 1024px
* 2048px

Then somebody requesting a 400px image might get the next size up, the
512px image delivered and scaled down in the browser. (On a high-resolution
display, you might fetch the 1024px image.)

In reality we'd want sizes that fit most common usage, and perhaps make
future markup & visual editor widgets promote using of standard sizes to
minimize the cases where you end up with something that's not an exact fit.


> * Mobile clients will want to minimize the amount of data transferred
>

This is a good reason for picking appropriate default sizes that would fit
with actual common usage.

Note that with SVG, SVG originals can be either much smaller or much larger
than a rasterized image -- in many cases we have SVGs that are much more
detailed than they need to be. So serving of SVG doesn't guarantee a
bandwidth save, though it can in well-designed cases.

Mobile also has the case that many (possibly even most in some markets)
devices have a greater than 1.0 device-to-CSS pixel ratio, so loading 1.5X
or 2.0X versions of raster images may be something we want in many cases.
In theory you could make a switch -- just as we have a 'disable images'
switch, we could make it a three-way control 'no images - low-resolution
images - high-resolution images'.

[And just to screw with people, Windows 8 / Windows RT is going with 1.4x
and 1.8x scaling factors instead of the 1.5x and 2.0x that Android and iOS
-- and Windows Phone 7 -- use. Fun huh! We're probably not going to have
exact scaled versions of them, they'll get the 1.5 or 2.0 and scale it down
a little probably.]

-- brion
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] gerrit search

[Markus Glaser]

Hi all,

I am trying to find a (any) html file in gerrit (trying to find out whether it 
is displayed as HTML in a browser, but that is not the issue here…). When I 
enter “html” in the search box, it apparently searches commit messages for html 
(?). When I try the search syntax as described here [1], I get an invalid query 
error. Is there any documentation on what search operators I can use or how the 
search works in general? Any hints are greatly appreciated :)

Cheers,
Markus

[1] http://gerrit.googlecode.com/svn/documentation/2.1.4/user-search.html
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] WLM Android App v1.2.2

[Philip Chang]

Greetings All,


We have updated the app on Google
Playwith
some important fixes and enhancements.

Here is a separate download link:

*http://dumps.wikimedia.org/android/WLM-v1.2.2.apk*


Changes in this version:

* Dynamically load campaign data - picks up campaign config changes without
requiring a new version of the app

* Camera and location features marked as optional to increase device
compatibility

* Localisation updates

* Increased accuracy when communicating with GPS satellites

* Disables 'Cancel upload' button when it is too late to cancel

* Bigger touch areas for checkboxes

* Moves {{Uploaded with WLM Mobile}} template to the source field on file
page

* Translated country name on monument page

* Fixed admin level display

* Strip bold/italic apostrophe markup from wikitext fields

* 'Back' after upload no longer returns you to upload progress

* Consistent photo upload comments

* Cache monuments in certain situations to reduce API calls

* Monument pin clustering disabled when zoomed in

* Lossless image compression


The app is now also available of the FOSS market, F-droid, at:

*http://f-droid.org/repository/browse/?fdid=org.wikipedia.wlm*



We are working on more fixes and enhancements, including a handy way to
upload more photos on Commons. This will make any mobile upload a
convenient placeholder for desktop uploads.

Please let us know if you have any issues by email or on the feedback
page
.

We have a new page for help with monument data issues:

https://www.mediawiki.org/wiki/Wiki_Loves_Monuments_mobile_application/Data_issues


Phil

-- 
Phil Inje Chang
Product Manager, Mobile
Wikimedia Foundation
415-812-0854 m
415-882-7982 x 6810
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l