Re: [Wikitech-l] Gerrit actively discourages discussion

[Yuri Astrakhan]

hehe, thanks! :), just pulled again, but, its broken a bit :(

https://gerrit.wikimedia.org/r/#/c/53131/ -- at first i thought it deleted
comment lines, but apparently its just in indecipherable colors, until
clicked


On Wed, Apr 3, 2013 at 2:49 PM, Jon Robson  wrote:

> Yuri
> Great minds think alike ;-)
>
> https://github.com/jdlrobson/gerrit-be-nice-to-me/commit/d6fd7913ff8fd7b1b57003797ba9ecb1134797d4
>
>
> On Wed, Apr 3, 2013 at 10:55 AM, Yuri Astrakhan  >wrote:
>
> > Jon, do you think it might make sense to hide all those jenkin-bot
> > comments? They are mostly noise, and the only time when compilation
> fails,
> > its ok to click to expand just that comment. Or maybe just expand the
> last
> > jenkin-bot comment, but hide all the previous ones.
> >
> >
> > On Wed, Apr 3, 2013 at 1:53 AM, Yuri Astrakhan  > >wrote:
> >
> > > This is awesome!!! Thanks Jon :) And no, I didn't look at the code, too
> > > scared :)
> > >
> > >
> > > On Wed, Apr 3, 2013 at 1:14 AM, Jon Robson 
> wrote:
> > >
> > >> I feel very dirty having done this but I made a chrome extension that
> > >> autoexpands all comments and adds a comment count next to unexpanded
> > older
> > >> patchsets.
> > >>
> > >> The code's horrible but it works - feel free to try it out:
> > >>
> > >> https://github.com/jdlrobson/gerrit-be-nice-to-me
> > >>
> > >> I suspect the best long term solution might be to rewrite the
> interface.
> > >> The fact it is so ajaxay makes enhancements hard unless someone
> actually
> > >> wants to get to know and work on the gerrit codebase themselves...
> > >>
> > >>
> > >> On Tue, Apr 2, 2013 at 7:54 PM, MZMcBride  wrote:
> > >>
> > >> > Christian Aistleitner wrote:
> > >> > >while I do not want to discourage discussion of those items on
> > >> > >wikitech-l, I nevertheless filed them in bugzilla, so we can keep
> > >> > >track of the issues / solutions.
> > >> >
> > >> > This is wonderful. Thank you for filing these bugs, Christian. I'll
> > >> take a
> > >> > look at them now.
> > >> >
> > >> > MZMcBride
> > >> >
> > >> >
> > >> >
> > >> > ___
> > >> > Wikitech-l mailing list
> > >> > Wikitech-l@lists.wikimedia.org
> > >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Jon Robson
> > >> http://jonrobson.me.uk
> > >> @rakugojon
> > >> ___
> > >> Wikitech-l mailing list
> > >> Wikitech-l@lists.wikimedia.org
> > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > >>
> > >
> > >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
>
> --
> Jon Robson
> http://jonrobson.me.uk
> @rakugojon
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit actively discourages discussion

[Yuri Astrakhan]

on both monitors i checked, 0.2 opacity is too low, plus I don't think it
should apply to the border, just the header inside it, which also removes
the need to restore it on click. What do you think?

function silentNoise( $commentPanel ) {
// makes jenkins comments less prominent
$commentPanel.find( '.commentPanelHeader' ).css( 'opacity', 0.6 );
}



On Wed, Apr 3, 2013 at 3:19 PM, Jon Robson  wrote:

> You'll need to reload the extension in chrome://extensions
> It purposely doesn't delete the comment lines - it just makes them
> invisible just in case you might want to see them. It's all one big hack so
> might be wrong but it works for me ;-)
>
> On Wed, Apr 3, 2013 at 12:17 PM, Yuri Astrakhan  >wrote:
>
> > hehe, thanks! :), just pulled again, but, its broken a bit :(
> >
> > https://gerrit.wikimedia.org/r/#/c/53131/ -- at first i thought it
> deleted
> > comment lines, but apparently its just in indecipherable colors, until
> > clicked
> >
> >
> > On Wed, Apr 3, 2013 at 2:49 PM, Jon Robson  wrote:
> >
> > > Yuri
> > > Great minds think alike ;-)
> > >
> > >
> >
> https://github.com/jdlrobson/gerrit-be-nice-to-me/commit/d6fd7913ff8fd7b1b57003797ba9ecb1134797d4
> > >
> > >
> > > On Wed, Apr 3, 2013 at 10:55 AM, Yuri Astrakhan <
> > yastrak...@wikimedia.org
> > > >wrote:
> > >
> > > > Jon, do you think it might make sense to hide all those jenkin-bot
> > > > comments? They are mostly noise, and the only time when compilation
> > > fails,
> > > > its ok to click to expand just that comment. Or maybe just expand the
> > > last
> > > > jenkin-bot comment, but hide all the previous ones.
> > > >
> > > >
> > > > On Wed, Apr 3, 2013 at 1:53 AM, Yuri Astrakhan <
> > yastrak...@wikimedia.org
> > > > >wrote:
> > > >
> > > > > This is awesome!!! Thanks Jon :) And no, I didn't look at the code,
> > too
> > > > > scared :)
> > > > >
> > > > >
> > > > > On Wed, Apr 3, 2013 at 1:14 AM, Jon Robson 
> > > wrote:
> > > > >
> > > > >> I feel very dirty having done this but I made a chrome extension
> > that
> > > > >> autoexpands all comments and adds a comment count next to
> unexpanded
> > > > older
> > > > >> patchsets.
> > > > >>
> > > > >> The code's horrible but it works - feel free to try it out:
> > > > >>
> > > > >> https://github.com/jdlrobson/gerrit-be-nice-to-me
> > > > >>
> > > > >> I suspect the best long term solution might be to rewrite the
> > > interface.
> > > > >> The fact it is so ajaxay makes enhancements hard unless someone
> > > actually
> > > > >> wants to get to know and work on the gerrit codebase themselves...
> > > > >>
> > > > >>
> > > > >> On Tue, Apr 2, 2013 at 7:54 PM, MZMcBride 
> wrote:
> > > > >>
> > > > >> > Christian Aistleitner wrote:
> > > > >> > >while I do not want to discourage discussion of those items on
> > > > >> > >wikitech-l, I nevertheless filed them in bugzilla, so we can
> keep
> > > > >> > >track of the issues / solutions.
> > > > >> >
> > > > >> > This is wonderful. Thank you for filing these bugs, Christian.
> > I'll
> > > > >> take a
> > > > >> > look at them now.
> > > > >> >
> > > > >> > MZMcBride
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > ___
> > > > >> > Wikitech-l mailing list
> > > > >> > Wikitech-l@lists.wikimedia.org
> > > > >> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Jon Robson
> > > > >> http://jonrobson.me.uk
> > > > >> @rakugojon
> > > > >> ___
> > > > >> Wikitech-l mailing list
> > > > >> Wikitech-l@lists.wikimedia.org
> > > > >> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > > >>
> > > > >
> > > > >
> > > > ___
> > > > Wikitech-l mailing list
> > > > Wikitech-l@lists.wikimedia.org
> > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > >
> > >
> > >
> > >
> > > --
> > > Jon Robson
> > > http://jonrobson.me.uk
> > > @rakugojon
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
>
> --
> Jon Robson
> http://jonrobson.me.uk
> @rakugojon
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Proposal: Wikitech contributors

[Yuri Astrakhan]

I think we should consolidate wikitech and mediawiki. If wikipedia.org can
fit all the world's knowledge, how come we can't fit all our technical
know-how on one site? The fragmentation is unnecessary and arbitrary, it
confuses most newcomers and those not paying attention to the "proper
separation" between them.

The site should highlight and share the technology platform used at
Wikimedia Foundation.
*We need a site for everyone to look at, participate in, and replicate our
technology.*
Replication could be anything from the most basic MW installation to a
complete replica of our cluster.

If we are not happy with the URL, lets come up with another one, or decide
that wikitech is the preferred name, and redirect everything there. The
main page would have mediawiki docs, labs, server info and stats,
hackathons, apis, RFCs, and everything else related. Any non-wiki
information like server health could be sub-domain of this site, possibly
with extra permissions. Lets create and share a knowledge platform, not
just software.

 :)


On Thu, Apr 4, 2013 at 3:24 AM, Quim Gil  wrote:

> Thank you for the vivid discussion about the potential future roles of
> http://mediawiki.org and http://wikitech.wikimedia.org
>
> I have updated http://www.mediawiki.org/wiki/**
> Technical_communications/Dev_**wiki_consolidation#Proposed_**solutionaccordingly.
>
>
> On 04/03/2013 03:51 PM, Steven Walling wrote:
>
>> who's going to do the work of
>> migrating all our project documentation over?
>>
>
> guillom, myself and whoever else wants to help.
>
> http://www.mediawiki.org/wiki/**Technical_communications/Dev_**
> wiki_consolidation#Steps
>
>
> --
> Quim Gil
> Technical Contributor Coordinator @ Wikimedia Foundation
> http://www.mediawiki.org/wiki/**User:Qgil
>
> __**_
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/**mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Adding messages to cache for wfMessage() at runtime

[Yuri Astrakhan]

Hi, I need to use the localization framework - wfMessage() to translate my
own custom messages. I have a message dictionary in the form of
{languageCode => message, ...}, and looking for a way to inject those
messages into the message cache. The actual messages dictionary is stored
as a custom setting as part of a json blob, and cannot be added to the
regular $message['xx'] = '...'; for  translatewiki.  Thanks!

Usage scenario:

// Array is generated from an external source and will look like this:
$messageMap = array('en'=>'aaa', 'fr'=>'b {{SITENAME}} b', ...);

// Some magic method to add the entire message map into message cache
messageCache->Add('custom-key1', $messageMap);

// Process message
// Note that in case 'fr' is not defined, it should fallback using default
resolution rules.
$result = wfMessage('custom-key1')->inLanguage('fr')->text();
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Adding messages to cache for wfMessage() at runtime

[Yuri Astrakhan]

> So, we used to have a $messagecache->add(), but it was removed
> after a very very long deprecation.
>

Just when  I was about to use it :) I basically look for this functionality:

* Figure out which localized string to use for language X, in case there is
no message in X. Something like "use Russian if Belarusian message is not
available". I remember this was present in pywiki framework, and suspect
mediawiki has it somewhere.

* Expand {{templates}}/PLURAL/GENDER/...
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Geohack tools

[Yuri Astrakhan]

Pine, you are right. That list is not very useful, and instead should look
something like this:

https://en.wikivoyage.org/wiki/Salzburg#Get_around

The bad news is that the tile service it uses is hosted on labs, which
means it cannot scale to the regular wikipedia-usage levels. Plus there
might be a potential policy problem there - default lab-content loading on
every page visit without user's consent.

The good news is that we are very close to launching a full-blown WMF
production-hosted tile service, based on the wonderful data from OSM.

See general info and some ideas people have proposed -
https://www.mediawiki.org/wiki/Maps  (feel free to add more)


On Fri, Aug 7, 2015 at 11:23 PM, Gergo Tisza  wrote:

> On Thu, Aug 6, 2015 at 11:01 PM, Pine W  wrote:
>
> > I just now realized how powerful these tools are when I started clicking
> > around.
> >
> >
> >
> https://tools.wmflabs.org/geohack/geohack.php?pagename=File%3AWhite-cheeked_Starling_perching_on_a_rock.jpg¶ms=34.610576_N_135.540542_E_globe:Earth_class:object_&language=en
> >
> > Is there any chance of integrating some of these tools more directly onto
> > Wikipedia pages, and into mobile web/mobile apps?
> >
>
> There is an ongoing project
>  to integrate
> maps
> into MediaWiki; that's probably the best place to catalog the use cases of
> Geohack and decide which can and need to be supported by the new tools.
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

[Yuri Astrakhan]

The STYLE of the map is in https://github.com/kartotherian/osm-bright.tm2
which is a fork from https://github.com/mapbox/mapbox-studio-osm-bright.tm2
-- released by Mapbox under a 3 clause BSD license

(IANAL).

On Thu, Sep 17, 2015 at 11:21 PM, Max Semenik  wrote:

> The underlying data is ODbL which is properly attributed per OSM
> guidelines. The license for tiles themselves is a different issue, OSM
> decided to CC them, we're not claiming any additional copyright so far,
> until legal tells us in https://phabricator.wikimedia.org/T111224
>
> On Thu, Sep 17, 2015 at 12:40 PM, Ryan Kaldari 
> wrote:
>
> > What is the license for the map tiles, i.e. the cartography rather than
> the
> > data? Open Street Maps uses CC-BY-SA, although they don't really make it
> > clear who is supposed to be attributed. It looks like our tiles are
> > different though.
> >
> > On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc 
> wrote:
> >
> > > The Discovery Department has launched an experimental tile and static
> > maps
> > > service available at https://maps.wikimedia.org.
> > >
> > > Using this service you can browse and embed map tiles into your own
> tools
> > > using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
> > .org
> > > and *.wikivoyage .org (referrer header must be either missing or set to
> > > these values) but we would like to open it up to Wikipedia traffic if
> we
> > > see enough use. Our hope is that this service fits the needs of the
> > > numerous maps developers and tool authors who have asked for a WMF
> hosted
> > > tile service with an initial focus on WikiVoyage.
> > >
> > > We'd love for you to try our new service, experiment writing tools
> using
> > > our tiles, and giving us feedback <
> > > https://www.mediawiki.org/wiki/Talk:Maps> .
> > > If you've built a tool using OpenStreetMap-based imagery then using our
> > > service is a simple drop-in replacement.
> > >
> > > Getting started is as easy as
> > > https://www.mediawiki.org/wiki/Maps#Getting_Started
> > >
> > > How can you help?
> > >
> > > * Adapt your labs tool to use this service - for example, use Leaflet
> js
> > > library and point it to https://maps.wikimedia.org
> > > * File bugs in Phabricator
> > > 
> > > * Provide us feedback to help guide future features
> > > 
> > > * Improve our map style <
> https://github.com/kartotherian/osm-bright.tm2>
> > > * Improve our data extraction
> > > 
> > >
> > > Based on usage and your feedback, the Discovery team
> > >  will decide how to proceed.
> > >
> > > We could add more data sources (both vector and raster), work on
> > additional
> > > services such as static maps or geosearch, work on supporting all
> > > languages, switch to client-side WebGL rendering, etc. Please help us
> > > decide what is most important.
> > >
> > > https://www.mediawiki.org/wiki/Maps has more about the project and
> > related
> > > Maps work.
> > >
> > > == In Depth ==
> > >
> > > Tiles are served from https://maps.wikimedia.org, but can only be
> > accessed
> > > from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> > Kartotherian
> > > can produce tiles as images (png), and as raw vector data (PBF Mapbox
> > > format or json):
> > >
> > > .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> > >
> > > Additionally, Kartotherian can produce snapshot (static) images of any
> > > location, scaling, and zoom level with
> > >
> > > .../{source},{zoom},{lat},{lon},{width}x{height}[@{scale}x].{format}.
> > >
> > > For example, to get an image centered at 42,-3.14, at zoom level 4,
> size
> > > 800x600, use
> > > https://maps.wikimedia.org/img/osm-intl,4,42,-3.14,800x600.png
> > > (copy/paste the link, or else it might not work due to referrer
> > > restriction).
> > >
> > > Do note that the static feature is highly experimental right now.
> > >
> > > We would like to thank WMF Ops (especially Alex Kosiaris, Brandon
> Black,
> > > and Jaime Crespo), services team, OSM community and engineers, and the
> > > Mapnik and Mapbox teams. The project would not have completed so fast
> > > without you.
> > >
> > > Thank You
> > >
> > > --tomasz
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
>
>
>
> --
> Best regards,
> Max Semenik ([[User:MaxSem]])
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

Re: [Wikitech-l] Announcing the launch of Maps

[Yuri Astrakhan]

The bracketing was introduced to reduce the DDOS attack surface and
increase the cache hit ratio. This means that we could in theory do all the
magic of bracketing in the varnish (to keep the cache rate high), or we
could force the clients to do the logic.  The problem with Varnish approach
is that it puts business-level logic into Varnish's brittle C-style
configs.

On Thu, Sep 17, 2015 at 11:58 PM, Brion Vibber 
wrote:

> Awesome sauce! Congrats to everyone who's been working on this.
>
>
> On Thu, Sep 17, 2015 at 12:26 PM, Tomasz Finc  wrote:
>
> > == In Depth ==
> >
> > Tiles are served from https://maps.wikimedia.org, but can only be
> accessed
> > from any subdomains of *.wmflabs .org and *.wikivoyage.org.
> Kartotherian
> > can produce tiles as images (png), and as raw vector data (PBF Mapbox
> > format or json):
> >
> > .../{source}/{zoom}/{x}/{y}[@{scale}x].{format}
> >
>
> Just a note -- the scale factor is for high-density displays, which are now
> standard on most phones and increasingly common on laptops (1.5x is common
> on small Windows laptops with 1080p displays). However the available scale
> factors are hardcoded, so you can't just take an arbitrary device scale
> factor (window.devicePixelRatio in HTML5) and feed it in, as it might not
> match and you'll get HTTP errors out.
>
> I've got a provisional patch in for the example front-end which manually
> brackets to the available settings:
> https://github.com/kartotherian/kartotherian/pull/13
>
> It'd be awesome to either expose the available scaling factors as a list
> via JSON API, or move the bracketing logic into the server side so you can
> send a query with your actual scale factor and get back one that will work.
>
> (If available scaling factors depend on the layer source being viewed, this
> might be more complicated.)
>
>
> > We would like to thank WMF Ops (especially Alex Kosiaris, Brandon Black,
> > and Jaime Crespo), services team, OSM community and engineers, and the
> > Mapnik and Mapbox teams. The project would not have completed so fast
> > without you.
> >
>
> Seconded. :)
>
> -- brion
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Announcing the launch of Maps

[Yuri Astrakhan]

Please file a phabricator task, and add maps and operations on it
On Sep 19, 2015 3:21 AM, "Yongmin Hong"  wrote:

> 2015. 9. 18. 오전 4:27에 "Tomasz Finc" 님이 작성:
> >
> > The Discovery Department has launched an experimental tile and static
> maps
> > service available at https://maps.wikimedia.org.
> >
> > Using this service you can browse and embed map tiles into your own tools
> > using OpenStreetMap data. Currently, we handle traffic from *.wmflabs
> .org
> > and *.wikivoyage .org (referrer header must be either missing or set to
> > these values) but we would like to open it up to Wikipedia traffic if we
> > see enough use. Our hope is that this service fits the needs of the
> > numerous maps developers and tool authors who have asked for a WMF hosted
> > tile service with an initial focus on WikiVoyage.
>
> It is sad that incubator.wikimedia.org is not included. (Wikivoyages not
> ready for their own wikis resides on this domain under prefix Wy/**, so
> Korean one becomes [[Wy/ko]].)
>
> - snip -
>
> --
> revi
> https://revi.me
> -- Sent from Android --
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Deprecate $wgEnableAPI?

[Yuri Astrakhan]

+1
On Oct 14, 2015 02:49, "Erik Bernhardson" 
wrote:

> +∞. This switch can only make things more complicated, and i highly doubt
> developers are taking this into consideration when writing new features. It
> seems like a 'please break everything' switch.
>
> On Tue, Oct 13, 2015 at 3:49 PM, Max Semenik 
> wrote:
>
> > Hey, I've created https://phabricator.wikimedia.org/T115414 to remove
> > $wgEnableAPI. Quoting the bug:
> >
> > By now, MediaWiki is severely crippled without the API, with most of
> > interesting extensions relying on the API, it becomes more and more
> > problematic to disable it. Now it's probably fair to say that wikis that
> do
> > that are just shooting their own feet off for no good reason. Therefore,
> I
> > propose to officially declare that MW does not support working without
> the
> > API and remove this very setting.
> >
> > Please state your opinions on this.
> >
> > --
> > Best regards,
> > Max Semenik ([[User:MaxSem]])
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] JetBrains IDE licenses for MW developers - php, js/node, python, C/C++, puppet, ruby, ...

[Yuri Astrakhan]

Developers, we now have licenses for JetBrains' IDEA Ultimate
 (JavaScript, PHP, Python, ...)  and CLion
 (C/C++).  The IDE supports debugging in
Vagrant and on-the-fly static code analysis.

If you are a volunteer developer, and want to use an IDE, contact me for a
license. Those with access to the office wiki may register directly at the
licenses  page.

Please try it for a bit before getting a license -- just in case you don't
like it.

P.S. This is only for those who want JetBrains' IDE. If you are happy with
Vim, good for you. If you like Atom or Sublime, great. If notepad is the
best text editor of all times, all the power to you.  Bikeshedding should
go into a separate thread ))
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Vision for Wikipedia

[Yuri Astrakhan]

I would like to share my vision for Wikipedia's future, as well as some
steps that could help us get there. I hope you find it useful. Please share
your feedback and ideas.

Vision: https://meta.wikimedia.org/wiki/User:Yurik/I_Dream_of_Content
Implementation:
https://meta.wikimedia.org/wiki/User:Yurik/From_Dream_to_Reality

Thanks!

P.S. The first link was previously shared on a different mailing list.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Flushing cached parser output after preview

[Yuri Astrakhan]

https://phabricator.wikimedia.org/T119779
Graph extension generates different html output depending on the isPreview
parser option, but if user previews a page and saves it right after without
any changes, the parser reuses previous output. Is there a way to force
parser regenerate on save? Thanks!
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Appreciation thread, 2015

[Yuri Astrakhan]

* JGirault, TheDJ, Milimetric, and Ferdbold for their hard work on graphs
* MaxSem, Akosiaris, bblack and johannesk_wmde for the maps

Everyone - for helping )

On Tue, Dec 8, 2015 at 2:38 AM, Isarra Yos  wrote:

> Perhaps a thanks to the people we don't think about, who we often never
> even work with. Volunteers and third-party developers who contribute to and
> maintain repositories not part of the big tickets. People who take the time
> to file bugs, and triage them, and sort things out. The ops who keep things
> running, even when on a plane. Those who comment on proposals, finding
> problems, expressing support. Liaisons, developers, designers who take the
> time to reach out and talk to the people affected by what they do, and take
> the time to listen.
>
> And also a thanks to everyone on this list for keeping things fairly
> sensible and productive and not turning it into a massive drama fest every
> tuesday.
>
>
> On 05/12/15 08:25, Legoktm wrote:
>
>> Hi,
>>
>> It's been a while since we had one of these, and 2015 has been a very
>> busy year! :-)
>>
>> I'll quote Sumana from last time:
>>
>> How about a little email thread for us to say nice things about each
>>> other?  Rules: be kind, thank someone, and say why you're thanking
>>> them.
>>> [If you don't get thanked and you feel mopey, email me and I'll
>>> comfort you. :-)]
>>>
>>
>> I'll start off with thank yous to:
>> * Addshore, for his awesome work on CatWatch and help with recent
>> ExtensionDistributor improvements
>> * YuviPanda, for coming up with unique and useful tools for Wikimedians,
>> and then following through and building them
>> * S Page, for consistently nagging me and other developers to improve
>> documentation
>>
>> -- Legoktm
>>
>> ___
>> Wikitech-l mailing list
>> Wikitech-l@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Interactive graphs & charts are now live

[Yuri Astrakhan]

The  tag has just been upgraded to Vega 2.0, adding interactivity
support.  See live examples here
.

Vega 1.0 is still available, but it is obsolete.  Please migrate all your
graphs to the new system so we can turn 1.0 off. See how to migrate:
https://github.com/vega/vega/wiki/Upgrading-to-2.0

Unless the graph definition contains a "version" value, the default is
still 1. After I tag all existing graphs, I will switch the default to
version 2.

See Vega main page for more: http://vega.github.io/vega/
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Wmfall] Community Wishlist Survey: Top 10 wishes!

[Yuri Astrakhan]

>
> #3. Central global repository for templates, gadgets and Lua modules  (87)
>

I would love to participate in this - I feel it would bring different
language communities much closer together.  And great for Graphs & Maps.


> #7. Pageview Stats tool  (70)
>

We could even do it on-wiki like here
 - it shows
how to draw a graph from pageviews API, and this graph extension
 can add
HTML controls to it.

Awesome list, thanks for getting more community involvement!
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Sharing JS code between NodeJS and browser

[Yuri Astrakhan]

For JS gurus - what is the best way to share JavaScript library code
between the NodeJS and browser?  The shared library will need to use
$.extend()-like functionality, URL parsing and reconstructing, and logging.

How should the code be organized? What are the recommended tools to
automate the process?

Thanks!
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Sharing JS code between NodeJS and browser

[Yuri Astrakhan]

Trevor and Daniel, thanks for your reply.

How would you structure the code that is to be shared?  Should it be a
separate NPM package, referenced from the extension package.json via git
url, and have a small file in the extension's lib/ dir with a oneliner -
 "require('...')"  that browserify could pick up? And have a script command
in package.json to build that file?

There are many implementations of URL parsing - from mw.Url to
require('url').  They differ slightly, but that is not the main problem -
how should the shared code use the right one?  Should they be passed as
functional function params? Should there be some "lib init" that sets lib's
global functions?

On Fri, Dec 18, 2015 at 8:22 PM, Daniel Friesen 
wrote:

> On 2015-12-18 8:41 AM, Trevor Parscal wrote:
> > Be careful when doing this with NPM modules, as their contents are
> subject
> > to change, and only their index file is configured and trying to
> > automatically know the paths and inclusion order is more of a mad science
> > than an art. Your best bet would be hard-coding and using very specific
> > versions in your package.json to protect from unexpected changes dues to
> > subtle NPM module version differences.
> npm modules are modules, not scripts. Even if you knew their execution
> order you can't just "include" them in any order at all.
>
> Using browserify to build a standalone script with a UMD that exposes a
> global might be the best bet.
> Though you'll have to be wary of requiring a module that pulls in a
> bunch of dependencies.
>
> > As for needing some $.extend, URL parsing and reconstructing, and
> logging.
> > I'm assuming you mean taking client code to the server where jQuery and
> > common browser functionally might not be present. There are many NPM
> > modules that provide shims for these things, including full-on
> server-side
> > jQuery. Usually the differences are subtle if any. Members of the Paraoid
> > team are very familiar with this space and are probably good people to
> talk
> > to about this.
> URL - Built in to node. Browserify comes with a browser compatible
> version of the module for when you require('url') it.
>
> extend:
> - lodash.merge (as part of lodash or the dedicated 'lodash.merge' module)
> - Some people use xtend or extend
> - There is also a new native Object.assign built into ES6, which you
> could include a simple polyfill for
>
> Logging varies very much by what you need. So there's wilson, debug, and
> a bunch of others.
>
> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] New tutorial for interactive graphics on Wiki

[Yuri Astrakhan]

I just finished writing a tutorial on how to build interactive Vega graphs
for Wikipedia. And yes, we could build video games this way too :)

https://www.mediawiki.org/wiki/Extension:Graph/Interactive_Graph_Tutorial
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] New tutorial for interactive graphics on Wiki

[Yuri Astrakhan]

Well, it already supports geo projections, with mouse-draggable map
spinning [1]... We could implement some 3d models if we can hide the
out-of-sight polygons :)

http://vega.github.io/vega-editor/?mode=vega&spec=map-params
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] New tutorial for interactive graphics on Wiki

[Yuri Astrakhan]

We could also use it for non-maps - basically the slider is a dynamic
parameter - and we could have multiple such parameters, and show data as
any kind of a picture - a chart, map, or even a state of animation.

On Fri, Jan 1, 2016 at 10:47 PM, Pine W  wrote:

> I've played with this more, and I like how it works. I can imagine people
> using this map with other data sets like HDI, GDP per capita, inflation,
> Wikipedia readership, and WMF fundraising (:
>
> Pine
>
> On Thu, Dec 31, 2015 at 2:13 AM, Szymon Grabarczuk <
> tar.locesil...@gmail.com
> > wrote:
>
> > Great! now, I think it should be marked for translation.
> >
> > On 31 December 2015 at 08:38, Pine W  wrote:
> >
> > > The tutorial looks nice! I will take a look when I am less distracted
> by
> > > finance reports. Thank you for working on this.
> > >
> > > Pine
> > > On Dec 30, 2015 22:37, "Yuri Astrakhan" 
> > wrote:
> > >
> > > > I just finished writing a tutorial on how to build interactive Vega
> > > graphs
> > > > for Wikipedia. And yes, we could build video games this way too :)
> > > >
> > > >
> > >
> >
> https://www.mediawiki.org/wiki/Extension:Graph/Interactive_Graph_Tutorial
> > > > ___
> > > > Wikitech-l mailing list
> > > > Wikitech-l@lists.wikimedia.org
> > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > > ___
> > > Wikitech-l mailing list
> > > Wikitech-l@lists.wikimedia.org
> > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> > >
> >
> >
> >
> > --
> > *Szymon Grabarczuk*
> >
> > Free Knowledge Advocacy Group EU
> > Head of Research & Development Group, Wikimedia Polska
> > pl.wikimedia.org/wiki/User:Tar_Lócesilion
> <http://pl.wikimedia.org/wiki/User:Tar_L%C3%B3cesilion>
> > <http://pl.wikimedia.org/wiki/User:Tar_L%C3%B3cesilion>
> > <http://pl.wikimedia.org/wiki/User:Tar_L%C3%B3cesilion>
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] New tutorial for interactive graphics on Wiki

[Yuri Astrakhan]

Florian, thank you!  Any help with the tutorial is greatly appreciated :)

On Sun, Jan 10, 2016 at 11:36 PM, Florian Schmidt <
florian.schmidt.wel...@t-online.de> wrote:

> Hi,
>
> I was free and edited the page:
> https://www.mediawiki.org/w/index.php?title=Extension%3AGraph%2FInteractive_Graph_Tutorial&type=revision&diff=2013042&oldid=2008358
>
> Hope that's a better wording, if not: feel free to edit it ;)
>
> Best,
> Florian
>
> -Original-Nachricht-
> Betreff: Re: [Wikitech-l] New tutorial for interactive graphics on Wiki
> Datum: 2016-01-10T20:21:40+0100
> Von: "Purodha Blissenbach" 
> An: "wikitech-l@lists.wikimedia.org" 
>
> Hi,
>
> in the 1st translation unit, there is a "klick here" type of link.
> There is a section explaining why that is a no-go:
> https://www.mediawiki.org/wiki/Localisation#Use_meaningful_link_anchors
>
> Purodha
>
> On 08.01.2016 18:49, Johan Jönsson wrote:> On Thu, Dec 31, 2015 at 11:13
> AM, Szymon Grabarczuk>  wrote:>> Great! now, I
> think it should be marked for translation.>> Agreed. And done.>> //Johan
> Jönsson> -->> ___> Wikitech-l
> mailing list> Wikitech-l@lists.wikimedia.org>
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Data visualization for wiki

[Yuri Astrakhan]

Ivo, there are two aspects of building graphs for Wiki:
* defining a "type" or "style" of a graph
* instantiating a graph with the specific data

Defining a new graph is a relatively complex programming task - one has to
program how graph would look like using Vega syntax. Instantiating is much
easier - one has to simply provide the needed data and a few parameters
controlling the graph.  Graph:Chart
  is a good example
which has been copied to many wikis.  The graph is generated via a complex
Lua script, but that complexity is hidden from all the graph users.

Frederic Bolduc (CCed) has done a good job building an experimental Visual
Editor plugin for graphs, but that plugin attempts to do both tasks -
define the new graph type and allow for users to instantiate it, and we do
not have enough resources to support both aspects well.  It would be ideal
if you helped with that extension to change it into a template editor +
galery picker. This way community may build the graphs it needs, and lists
them in a "gallery of good graphs". Visual Editor would than show a "list
of recommended graphs" as a set of icons. Editors will be able to quickly
insert a graph template and provide data for it.

On Fri, Feb 5, 2016 at 2:10 AM, Ivo Kruusamägi 
wrote:

> So, some questions:
>
> * how important could that in-wiki image editing capability be?
> * has anything been done so far?
>
> I'd like to find some topic suitable for university student to work on. So
> here it is: free workforce for wiki. Act now the get that! Offer some
> topics. PHP related tasks preferred.
>
> Ivo
>
> 2016-02-04 14:20 GMT+02:00 Ivo Kruusamägi :
>
> > OK, this Graph extension seems rather advanced and even though not
> exactly
> > what I had in mind, somewhat similar. So may I conclude it would be
> > pointless to go into that direction? Or are there some things, that may
> > need someone to work with them? Like (making random example): building
> very
> > simple Visual Editor add-on to enable fast creation of some more simple
> > graphs (like pie charts from the enormous variety of possibilities
> > )?
> >
> > Could I get more information about need for image editors? *I'd like to
> > find something that might be suitable topic for bachelor thesis* and
> > vandering around alone inside the topic not yet known to me may not prove
> > to be fruitful.
> >
> > As for this Commons topic. Yes, something similar to Wikidata Game, but
> > directed towards Commons and not Wikidata. More in line with Estonian
> sites
> > Ajapaik  (for adding geolocation information
> > to old images), sift.pics (for determining image types) and
> > postkaart.ajapaik.ee (for digitalizing text in old postcards).
> >
> > Ivo
> >
> > 2016-02-04 5:00 GMT+02:00 MZMcBride :
> >
> >> Ivo Kruusamägi wrote:
> >> >What I'd like to see is some development, that would make it possible
> for
> >> >user to create visualizations inside MediaWiki. Something so easy that
> a
> >> >child could do that. Like this . Workflow example:
> >> 1)
> >> >user selects sth like Create Data Visualization, 2) has some selections
> >> >about cart type, colors, etc, 3) place to write down text (title, axes,
> >> >description) and 4) a table to fill in with data (values + their text
> >> >labels). That could then be saved as one revision. After that every
> other
> >> >user could edit this graph with the same selections and data tables
> just
> >> >like users edit articles and edit history is saved and easy to compare.
> >> >
> >> >Image files like this
> >> > or that
> >> ><
> >>
> https://commons.wikimedia.org/wiki/File:Tree_map_exports_2010_Estonia.svg
> >> > are ridiculous and fixes like that
> >> > are not that
> >> flexible,
> >> >pretty and easy to use as what we need. So lets move forward.
> >>
> >> Are you familiar with the "Graph" MediaWiki extension? Here's a demo:
> >> . This MediaWiki
> >> extension is deployed to Wikimedia wikis, including all the Wikipedias.
> >>
> >> >There are plenty of GPL licensed solutions that could be integrated
> with
> >> >MediaWiki. But I can't be only one thinking about this.
> >>
> >> You're not. :-)
> >>
> >> >So what should I know about that topic so that this work could really
> be
> >> >useful? I.e. how to avoid reinventing the wheel (like building
> something
> >> >already in development) and how to be sure that it could be easily
> >> >incorporated later? Who would be the perfect people to talk about this
> >> >topic?
> >>
> >> This mailing list is great for general discussion. Or we have a
> >> Phabricator installation at  where
> we
> >> track bugs and feature requests. You can search around for Phabricator
> >> Maniphes

Re: [Wikitech-l] T43327: Add page views graph(s) to MediaWiki's info action for Wikimedia wikis

[Yuri Astrakhan]

he.wiki has already enabled it by inserting Graph:PageViews template into
the info template, but yours does look cleaner. The graph will show only if
you did not modify your interface language in the Hebrew wiki, or if you
are not logged in.

https://he.wikipedia.org/w/index.php?title=%D7%A2%D7%9E%D7%95%D7%93_%D7%A8%D7%90%D7%A9%D7%99&action=info

One thing that concerns me though - you seem to get all the Vega and d3
code and graph data during the page load, yet you only show the graph on
click. Seems to take a lot of extra time and very wasteful. Also, it
doesn't work for older browsers.

On Fri, Mar 11, 2016 at 4:33 AM, Legoktm 
wrote:

> Hi!
>
> (I meant to send this out earlier but got distracted by other things :()
>
> At the Wikimedia developer summit, Addshore and I started working on a
> MediaWiki extension to resolve T43327[1], which asked for page view
> graphs to be added to action=info. The "WikimediaPageViewInfo"[2]
> extension has passed a security review, and should be deployed soon™.
>
> If you want to try it out, there's a test wiki[3] which uses the
> pageview data for en.wp.
>
> And for updates on the deployment status, please follow T125917[4].
>
> [1] https://phabricator.wikimedia.org/T43327
> [2] https://www.mediawiki.org/wiki/Extension:WikimediaPageViewInfo
> [3] http://bf-wmpageview.wmflabs.org/wiki/Taylor_Swift?action=info
> [4] https://phabricator.wikimedia.org/T125917
>
> -- Legoktm
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] X-Wikimedia-Debug, your new secret side-kick

[Yuri Astrakhan]

Isn't there a recommendation not to use the X- prefix for any new headers?
On Mar 31, 2016 12:23 PM, "Jaime Crespo"  wrote:

> On Thu, Mar 31, 2016 at 3:32 AM, Ori Livneh  wrote:
>
> > Cool? Cool.
>
> Definitely Cool.
>
> --
> Jaime Crespo
> 
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] JetBrains licenses for Volunteers and Staff

[Yuri Astrakhan]

PhpStorm, InteliJ IDEA, Resharper and other JetBrain users, we just
received free upgraded licenses for all of their products.

Check your account at https://account.jetbrains.com/licenses and login with
that account inside your application to automatically use that license. If
you don't see the license, contact me or Sam Reed, and we will add you
right away.  On IRC: yurik or reedy, or via email.  We could also use a few
more admins for these licenses. You will no longer need to copy/paste any
licenses manually.

IDEA:  use for PHP, JavaScript, Puppets, Ruby, and other web-related
technologies
https://www.jetbrains.com/idea/

Resharper: for C# and Microsoft Visual Studio C++
https://www.jetbrains.com/resharper/

CLion: for C++
https://www.jetbrains.com/clion/
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] JetBrains licenses for Volunteers and Staff

[Yuri Astrakhan]

Hi Pine,

They are not upgrades, they are full one year renewable free licenses for
the powerful ide with php, js, puppet, Ruby, Python, c++, ... , donated to
help with Wiki development.

You basically need to be a volunteer or a staff developer working on Wiki
projects with some history of code contributions.   Resharper is a great
product and it is part of the license, but I am not sure we have any .net
code (I think we used to run bits of mono code somewhere, but that was a
while ago). Have you submitted any patches? In what languages?

Thanks!
On May 10, 2016 08:48, "Pine W"  wrote:

> Hi Yuri,
>
> ReSharper interests me. This email refers to license upgrades. What is
> involved in initially qualifying for a license?
>
> Thanks,
>
> Pine
> On Apr 18, 2016 09:44, "Yuri Astrakhan"  wrote:
>
> > PhpStorm, InteliJ IDEA, Resharper and other JetBrain users, we just
> > received free upgraded licenses for all of their products.
> >
> > Check your account at https://account.jetbrains.com/licenses and login
> > with
> > that account inside your application to automatically use that license.
> If
> > you don't see the license, contact me or Sam Reed, and we will add you
> > right away.  On IRC: yurik or reedy, or via email.  We could also use a
> few
> > more admins for these licenses. You will no longer need to copy/paste any
> > licenses manually.
> >
> > IDEA:  use for PHP, JavaScript, Puppets, Ruby, and other web-related
> > technologies
> > https://www.jetbrains.com/idea/
> >
> > Resharper: for C# and Microsoft Visual Studio C++
> > https://www.jetbrains.com/resharper/
> >
> > CLion: for C++
> > https://www.jetbrains.com/clion/
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Enabling shared tabular data pages

[Yuri Astrakhan]

We have had some good feedback for the new shared tabular data feature, and
we are getting ready to deploy it in production. It would be amazing if you
can give it a final look-over to see if there are any blockers left.

The first stage will be to enable  Data:*.tab  pages on Commons, and allow
all other wikis direct access to it via Lua code and Graph extension. All
data at this point must be licensed under CC0. More licensing options are
still under discussion, and can be easily added later.

In line with the "release early, release often", we will not have any
elaborate data editing interface beyond the raw JSON code editor for the
first release. Our initial target audience is the more experienced users
who will evaluate and test the new technology. Once the underlying tech is
stable and prooven, we will work on making it more accessible to the
general audience.

Links:
* Task: https://phabricator.wikimedia.org/T134426
* Demo: http://data.wmflabs.org
* Technical: https://www.mediawiki.org/wiki/Extension:JsonConfig/Tabular
* Discussion:
https://commons.wikimedia.org/wiki/Commons:Village_pump/Proposals#Tabular_data_storage_for_Commons.21
* Facebook:
https://www.facebook.com/groups/wikipediaweekly/permalink/997545366959961/
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Enabling shared tabular data pages

[Yuri Astrakhan]

Daniel, I agree about the data/api versioning. I was mostly talking about
features and capabilities. For example, we could spend the next year
developing a visual table editor, implement support for unlimited table
sizes, provide import/export from other table formats, introduce elaborate
schema validation, and many other cool features. And after that year
realize that users don't need this whole thing at all, or need something
similar but very different.  Or we could release one small, well defined,
stable subset of that functionality, get feedback, and move forward.

Do you have any thoughts about the proposed data structure?

On Mon, Jun 6, 2016 at 4:09 PM, Daniel Kinzler 
wrote:

> Am 04.06.2016 um 18:47 schrieb Yuri Astrakhan:
> > In line with the "release early, release often", we will not have any
> > elaborate data editing interface beyond the raw JSON code editor for the
> > first release.
>
> A word of caution about this strategy: this is great for user facing
> things, but
> it really sucks if you are creating artifacts, such as page revisions. You
> will
> have to stay compatible with your very first data format, and the second,
> and
> the third, etc, forever. Similarly, once you have an ecosystem of tools
> that
> rely on your API and data model, changing it becomes rather troublesome.
>
> So, for anything that is supposed to offer a stable API, or creates
> persistent
> data, "release early, release often" is not a good strategy in my
> experience. A
> lot of pain lies this way. Remember: wikitext syntax was once a "let's
> just make
> it work, we will fix it later" hack...
>
> -- daniel
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Enabling shared tabular data pages

[Yuri Astrakhan]

Daniel, thanks, inline:

The structure looks sane and future-proof to me, but since it's
> all-in-one-blob,
> it'll be hard to scale it to more than a few ten thousand lines or so. I
> like
> this model, but if you want to go beyond that (DO we want to go beyond
> that?!)
> you will need a different approach, which may be incompatible.
>

We do *eventually* want to go beyond that towards large data. We had this
discussion with Brion, see here:
*  https://phabricator.wikimedia.org/T120452#2224764

I do not think my approach is a blocker for larger datasets, because you
can add simple SQL-like interface capable of reading data from these pages
and from large backend databases. 2MB page limit will prevent page data
from growing too large. Also, larger datasets is a different target, that
we should approach when we are ready.

One thing that should be specified very rigorously from the start are the
> supported data types, along with their exact syntax and semantics. Your
> example
> has string, number, boolean, and localized. So:
>
> * what's the length limit for string?
>
Good question. Do you have a limit for Wikidata labels and other string
values?

> * what's the range and precision of number? Is it the same as for JSON?
>
For now, same as JSON.

> * does boolean only accept JSON primitives, or also strings?
>
true/false only, no strings

> * what language codes are valid for localized? Is language fallback
> applied for
> display?
>
Same rules as for wiki language codes (but without validation against the
actual list). Automatic fallback is already implemented, using Language
class.  If everything else fails, and there is no English, takes random
first (unlike Language which stops at English and fails otherwise).


> You write in your proposal "Hard to define types like Wikidata ID,
> datetime, and
> URL could be stored as a string until we can reuse Wikidata's type system".
> Well, what's keeping you from using it now? DataValue and friends are
> standalone
> composer modules, you can find them on github.

I was told by the Wikidata team at the Jerusalem hackathon that the
Javascript code is too entangled, and I won't be able to reuse it for
non-Wikidata stuff.  I will be very happy to adapt it if possible. Yet, I
do not think this is a requirement for the first release.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Enabling shared tabular data pages

[Yuri Astrakhan]

Rob, thanks for your offer to help! Always welcome :)

By discussion and positive feedback I meant Facebook and Commons comments,
and a very old and elaborate phab ticket discussion:
* https://phabricator.wikimedia.org/T120452
*
https://commons.wikimedia.org/wiki/Commons:Village_pump/Proposals#Tabular_data_storage_for_Commons.21
* https://www.facebook.com/groups/wikipediaweekly/permalink/997545366959961/

I do not think this feature will immediately have a huge uptake. It will
simplify graph design, as it will be possible to store data outside of the
graph. It will also allow data from tables and lists to be moved into
separate wiki pages. The work of moving existing data into these pages
might not be very fast. In short, it will only be accessible from Lua and
graphs, will be less than 2MB each, and will require some technical skills
to edit JSON until better tools are created.


On Mon, Jun 6, 2016 at 9:14 PM, Rob Lanphier  wrote:

> On Mon, Jun 6, 2016 at 6:40 AM, Yuri Astrakhan 
> wrote:
>
> > Daniel, I agree about the data/api versioning. I was mostly talking about
> > features and capabilities. For example, we could spend the next year
> > developing a visual table editor, implement support for unlimited table
> > sizes, provide import/export from other table formats, introduce
> elaborate
> > schema validation, and many other cool features. And after that year
> > realize that users don't need this whole thing at all, or need something
> > similar but very different.  Or we could release one small, well defined,
> > stable subset of that functionality, get feedback, and move forward.
> >
>
> Hi Yuri,
>
> I think one thing that would be helpful for me (and I suspect many people
> who want to help) is some more specifics about this statement from your
> original email: "We have had some good feedback for the new shared tabular
> data feature, and we are getting ready to deploy it in production."  Which
> "we" are you referring to, and by "getting ready to deploy it in
> production", does that mean it's about to be usable where someone could
> upload gigabytes of production data in this format Commons by the end of
> the week?  Is there a more measured plan published somewhere?
>
> This all sounds very cool, but also an area where we could accidentally
> accrue a crushing load of technical debt without fully realizing it (per
> Daniel's comment).  I'll confess to being ignorant on everything that's
> been going on, and I'm wondering now how desperately I should study your
> documentation to make up for it (and how important it is to drop other work
> to make time for this).
>
> Rob
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] 2016W24 ArchCom RFC meeting (2016-06-15)

[Yuri Astrakhan]

Thank you Pine for your support! I do hope we can get this sorted out and
deployed :)

On Wed, Jun 15, 2016 at 10:30 AM, Pine W  wrote:

> I can't attend this meeting, but want to continue to voice support for
> improving MediaWiki (primarily VE) and Commons ability to accept,
> manipulate, and display spreadsheets and their kin.
>
> Thanks for working on this.
>
> Pine
> On Jun 15, 2016 00:11, "Rob Lanphier"  wrote:
>
> > Hi everyone,
> >
> > We're holding another ArchCom-RFC meeting this week to follow up on
> > the thread Yuri started about T120452.
> >
> > The ArchCom status page is finally up-to-date with last week's info:
> > 
> >
> > (and it's quoted below)
> >
> > Links!:
> > This week's meeting: 
> > RFC for this week's meeting: 
> > Subject of the RFC: Allow tabular datasets on Commons (or some similar
> > central repository) (CSV, TSV, JSON, XML)
> > Location: #wikimedia-office IRC channel
> > Time: 2016-06-15 Wednesday 21:00 UTC (2pm PDT, 23:00 CEST)
> >
> > Meetbot looks forward to scribing your presence!
> >
> > Rob
> >
> >
> > 
> > Source of [[mw:Architecture_committee/Status]], where "phab:" links
> > are pointers to https://phabricator.wikimedia.org
> >
> > This page provides status update for [[Requests for
> > comment|ArchCom-RFCs]], with an emphasis on ArchCom team member.  As
> > of this writing on 2016-04-29, this update is an experiment discussed
> > [[Topic:T2zctt083izvx07l|weekly ArchCom update discussion on the
> > "ArchCom/Team practices" talk page]].
> >
> > = Recent RFC meetings =
> > *ArchCom Planning meeting 2016W23: 2016-06-08: [[Phab:E202]] (E156/10)
> > **Notes: [[Architecture committee/2016-06-08]]
> > *ArchCom-RFC office hour 2016W23: 2016-06-08: [[Phab:E203]] (E66/38)
> > ** [[Phab:T89331|T89331 Replace Tidy in MW parser with HTML 5
> > parse/reserialize]]
> >
> > = Upcoming RFC meetings =
> > *ArchCom Planning meeting 2016W24: 2016-06-15: [[Phab:E212]] (E156/11)
> > **Notes: [[Architecture committee/2016-06-15]]
> > *ArchCom-RFC office hour 2016W24: 2016-06-15: [[Phab:E213]] (E66/39)
> > ** [[Phab:T120452|T120452]]: Allow tabular datasets on Commons (or
> > some similar central repository) (CSV, TSV, JSON, XML)
> > *** See also: comments in T124569 and T134426
> >
> > = Entering Final Comment Period =
> > * None.
> >
> > = Recently Approved =
> > * none
> >
> >  RFC inbox 
> > * [[phab:tag/archcom-rfc/|ArchCom RFC board]]:
> > ** [[Phab:T124569|T124569 RFC: Data namespace blob storage on
> wikidata.org
> > ]]
> >
> > = Shepherd status =
> > * Brion
> > ** [[Phab:T107595|T107595]] Multi-content revisions is interesting,
> > needed for various things in multimedia land
> > *** Meeting happened earlier this week; notes are on the ticket
> > ** T66214 - predictable thumb URLs
> > *** Break this out into:
> >  Define set of core & extensible media file options for Handler
> > extensions
> >  Predictable thumb URLs
> >  Improve InstantCommons perf by reducing need to run thumbnail URL
> > lookups
> >  Iframe-based rich media embedding for InstantCommons
> > ** plan to write up new RfCs for:
> > *** In-browser SVG rendering (pick up existing bug & mailing list notes)
> > *** iframe+CSP-isolated JS widgets for rich content
> >  & extend that to InstantCommons via embedding
> > *** iframe+CSP-isolated JS gadgets for UI plugins
> >  Build these out from ideas from hackathon project T131436
> > * Daniel
> > ** Software Quality working group - will follow up on earlier
> > proposal.  Will talk to people at Wikimania
> > ** Working on Multi Content Rev Spec with Brion
> > ** T113034 [[phab:T113034|RFC: Overhaul Interwiki map, unify with
> > Sites and WikiMap]]: checking in with Adam
> > ** T89733 (approved, with Stas driving implementation)
> > * Gabriel
> > ** Looking into content composition working group, possibly kick-off
> > at Wikimania
> > ** Discussing Multi Content Rev / RB interaction with Daniel;
> > follow-up at Wikimania
> > * Roan
> > ** T108655 [[phab:T108655|RFC: Standardise JavaScript interfaces]]: I
> > need to start the second part, but the recent comments have me
> > confused. I'll need to talk to Timo and figure out what the subject of
> > part two should be.
> > * RobLa
> > ** Working with [[User:DPatrick (WMF)|DPatrick]] on [[Wikimedia
> > Security Team]] issues in an attempt to be useful there.
> > ** T123753 [[phab:T123753|Establish retrospective reports for Security
> > and Performance incidents]]
> > *** In scope for this group?
> > ** Forming ArchCom-affiliated working groups
> > *** RFCs
> >  T124504 [[phab:T124504|Transition WikiDev '16 working areas into
> > working groups]] and
> >  T123606 [[Phab:T123606|RFC: Implement ArchCom-affiliated working
> > groups (process inspired by Rust's "subteams")]]
> > *** Testing Conpherence use

[Wikitech-l] to be enabled tomorrow

[Yuri Astrakhan]

Seems we have no more blockers for  tag on all Wikipedias. We will
enable it tomorrow, Sept 1st.  tag allows editors to add a link to
a popup map, complete with extra geojson overlay data.

See help page for instructions:
https://www.mediawiki.org/wiki/Help:Extension:Kartographer#.3Cmaplink.3E

Submit bugs to phabricator:
https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=Kartographer,maps
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] to be enabled tomorrow

[Yuri Astrakhan]

We tried to deploy it yesterday, but ran into some issues, so will try
again next week. Please keep an eye on
https://phabricator.wikimedia.org/T144062

On Fri, Sep 2, 2016, 12:03 Strainu  wrote:

> Hi Yuri,
>
> This does not seem to have happened. Is there a new ETA for this?
>
> Thanks,
>Andrei
>
> 2016-08-31 23:19 GMT+03:00 Yuri Astrakhan :
> > Seems we have no more blockers for  tag on all Wikipedias. We
> will
> > enable it tomorrow, Sept 1st.  tag allows editors to add a link
> to
> > a popup map, complete with extra geojson overlay data.
> >
> > See help page for instructions:
> > https://www.mediawiki.org/wiki/Help:Extension:Kartographer#.3Cmaplink.3E
> >
> > Submit bugs to phabricator:
> >
> https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=Kartographer,maps
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Wikipedia gets Map links and Geoshapes service

[Yuri Astrakhan]

Dear community, this week we enabled  support on all Wikipedia and
sister projects.  This means that now an article can have a link to a map,
and that map may contain highlighted regions and popups with information.
[1],[3]

Our next step is to add an informational sidebar to the map, similar to
what is being shown on the "geohack" page (map link in the upper right
corner of most location articles).  Check out proposed screenshots [2]

We now also have a geoshapes service. So if Open Street Maps community has
defined a region and assigned it a Wikidata ID, you can draw it on the map
with that ID. Or you can use Wikidata Query Service (via SPARQL language),
to query for those IDs and draw them on the map, coloring them and adding
popup information. [3] Geoshapes can also be used for the graphs. [4]


[1] Maps docs: https://www.mediawiki.org/wiki/Help:Extension:Kartographer
[2] sidebar prototype: https://phabricator.wikimedia.org/T131907#2616275
[3] US Governors
https://en.wikipedia.org/wiki/User:Yurik/maplink#/maplink/0
[4] Graph geoshapes
https://www.mediawiki.org/wiki/Template:Graph:Country_with_regions_and_capitals
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Shared localized maps data demo

[Yuri Astrakhan]

I would like to show one of the projects that Interactive team has been
hacking on: localizable maps data (GeoJSON), stored on Commons, and usable
from multiple wikis. I hope we can get it polished and enabled in
production soon enough - so far, lab's beta cluster only:

https://en.wikipedia.beta.wmflabs.org/wiki/Maplink-page
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Localizable data for Graphs and Templates on Commons

[Yuri Astrakhan]

Following the localizable maps example, here is a structured tabular data
example that also supports localization, shared data, and can be used
directly from the graphs or from Lua scripts on any wiki. Note that the
graph itself is in English wiki (labs), but data comes from Commons. Feel
free to add translations.

English:
https://en.wikipedia.beta.wmflabs.org/wiki/Dimpvis_-_Fertility_vs_Life_Expectancy
Russian:
https://en.wikipedia.beta.wmflabs.org/wiki/Dimpvis_-_Fertility_vs_Life_Expectancy?uselang=ru



On Mon, Nov 7, 2016 at 11:46 PM Yuri Astrakhan 
wrote:

> I would like to show one of the projects that Interactive team has been
> hacking on: localizable maps data (GeoJSON), stored on Commons, and usable
> from multiple wikis. I hope we can get it polished and enabled in
> production soon enough - so far, lab's beta cluster only:
>
> https://en.wikipedia.beta.wmflabs.org/wiki/Maplink-page
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Arbitrary Wikidata querying

[Yuri Astrakhan]

AFAIK, you can query data from Wikidata, but you cannot put it into a page,
unless its a graph. Graphs can do it -
https://www.mediawiki.org/wiki/Extension:Graph/Demo/Sparql

As of last Thursday, you can also create a table on Commons Data namespace,
and make a simple Lua script on your favorite wiki to pull that data in and
render it. Since Wikidata is accessible from Lua, you could pull useful
information about each president. I am not sure about the efficiency
aspects here.

WeatherDemo  -- pulls
data from commons Data:Weather/New_York_City.tab
 and
formats it using enwiki module
.  I'm still working on
some fun demos for a big presentation.


On Sat, Dec 10, 2016 at 8:30 PM MZMcBride  wrote:

> Hi.
>
> If I wanted to make a page on the English Wikipedia using wikitext called
> "List of United States presidents" that dynamically embeds information
> from  and
>  and other similar items, is this
> currently possible? I consider this to be arbitrary Wikidata querying, but
> if that's not the correct term, please let me know what to call it.
>
> A more advanced form of this Wikidata querying would be dynamically
> generating a list of presidents of the United States by finding every
> Wikidata item where position held includes "President of the United
> States". Is this currently possible on-wiki or via wikitext?
>
> If either of these querying capabilities are possible, how do I do them?
> I don't understand how to query Wikidata in a useful way and I find this
> frustrating. Since 2012, we've been putting a lot of data into Wikidata,
> but I want to programmatically extract some of this data and use it in my
> Wikipedia editing. How do I do this?
>
> If these querying capabilities are not currently possible, when might they
> be? I understand that cache invalidation is difficult and that this will
> need a sensible editing user interface, but I don't care about all of
> that, I just want to be able to query data out of this large data store.
>
> MZMcBride
>
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Now live: Shared structured data

[Yuri Astrakhan]

Gift season! We have launched structured data on Commons, available from
all wikis.

TLDR; One data store. Use everywhere. Upload table data to Commons, with
localization, and use it to create wiki tables, lists, or use directly in
graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this
per-state GDP map demo, and select multiple years. More demos at the bottom.
US Map state highlight


Data can now be stored as *.tab and *.map pages in the data namespace on
Commons. That data may contain localization, so a table cell could be in
multiple languages. And that data is accessible from any wikis, by Lua
scripts, Graphs, and Maps.

Lua lets you generate wiki tables from the data by filtering, converting,
mixing, and formatting the raw data. Lua also lets you generate lists. Or
any wiki markup.

Graphs can use both .tab and .map directly to visualize the data and let
users interact with it. The GDP demo above uses a map from Commons, and
colors each segment with the data based on a data table.

Kartographer (/) can use the .map data as an extra layer
on top of the base map. This way we can show endangered species' habitat.

== Demo ==
* Raw data example

* Interactive Weather data

* Same data in Weather template

* Interactive GDP map

* Endangered Jemez Mountains salamander - habitat

* Population history

* Line chart 

== Getting started ==
* Try creating a page at data:Sandbox/.tab on Commons. Don't forget
the .tab extension, or it won't work.
* Try using some data with the Line chart graph template
A thorough guide is needed, help is welcome!

== Documentation links ==
* Tabular help 
* Map help 
If you find a bug, create Phabricator ticket with #tabular-data tag, or
comment on the documentation talk pages.

== FAQ ==
* Relation to Wikidata:  Wikidata is about "facts" (small pieces of
information). Structured data is about "blobs" - large amounts of data like
the historical weather or the outline of the state of New York.

== TODOs ==
* Add a nice "table editor" - editing JSON by hand is cruel. T134618
* "What links here" should track data usage across wikis. Will allow
quicker auto-refresh of the pages too. T153966
* Support data redirects. T153598
* Mega epic: Support external data feeds.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Now live: Shared structured data

[Yuri Astrakhan]

Yes, there seem to have been a bit of a naming collision.  Tabular data and
map data have been jointly known as structured data, but there is also the
Structured Data project, which IMO should be called Structured Metadata
project :)  Naming suggestions are welcome!

P.S. Brad, I'm sorry tabular and map data did not excite you :(

On Thu, Dec 22, 2016 at 2:38 PM Brad Jorsch (Anomie) 
wrote:

> On Thu, Dec 22, 2016 at 2:30 PM, Yuri Astrakhan 
> wrote:
>
> > Gift season! We have launched structured data on Commons, available from
> > all wikis.
> >
>
> I was momentarily excited, then I read a little farther and discovered this
> isn't about https://commons.wikimedia.org/wiki/Commons:Structured_data.
>
>
> --
> Brad Jorsch (Anomie)
> Senior Software Engineer
> Wikimedia Foundation
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Now live: Shared structured data

[Yuri Astrakhan]

Micru, thanks, I think Datasets sounds like a good name too!

On Thu, Dec 22, 2016 at 2:44 PM David Cuenca Tudela 
wrote:

> On Thu, Dec 22, 2016 at 8:38 PM, Brad Jorsch (Anomie) <
> bjor...@wikimedia.org
> > wrote:
>
> > On Thu, Dec 22, 2016 at 2:30 PM, Yuri Astrakhan <
> yastrak...@wikimedia.org>
> > wrote:
> >
> > > Gift season! We have launched structured data on Commons, available
> from
> > > all wikis.
> > >
> >
> > I was momentarily excited, then I read a little farther and discovered
> this
> > isn't about https://commons.wikimedia.org/wiki/Commons:Structured_data.
> >
>
> Same here, I think it needs a better name...
>
> What about calling it datasets or structured datasets?
>
> Cheers,
> Micru
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [discovery] Now live: Shared structured data

[Yuri Astrakhan]

Svetlana, thanks for suggestion. I think we should create a portal similar
to the Structured Data one, and put some examples there.  Deciding on the
name is difficult :)   "Commons Datasets" does sound good.

There has been a very prolonged discussion on where to host this feature -
https://meta.wikimedia.org/wiki/User:Yurik/Storing_data. Wikidata would
have been a good choice, but users expect all the data there to be in
public domain, and we may add more licensing choices later.

> An inline example with English commentary -- straight on the first page about
this new technology without making users click links -- could be nice. The
text you typed up does not seem to be on a wiki page, so I am unable to
edit it...

Which page are you referring to?

On Thu, Dec 22, 2016 at 4:03 PM Svetlana Tkachenko 
wrote:

> Hello,
>
> Maybe 'commons store' or 'commons datasets' could work? I would suggest
> that the name reflects on the fact that the datasets are shared
> ('common') and are not on Wikidata.
>
> If I may ask, why is it in commons.wikimedia.org/wiki/Data:* and not at
> Meta (like Global user pages) or Wikidata (like structured data about
> lots of things)?
>
> An inline example with English commentary -- straight on the first page
> about this new technology without making users click links -- could be
> nice. The text you typed up does not seem to be on a wiki page, so I am
> unable to edit it...
>
> Svetlana.
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Now live: Shared structured data

[Yuri Astrakhan]

Hi Mathieu, yes, I think you can totally build up this glossary in a
dataset. Just remember that each string can be no longer then 400 chars,
and total size under 2mb.

On Sun, Dec 25, 2016, 10:45 mathieu stumpf guntz <
psychosl...@culture-libre.org> wrote:

> Hi Yuri,
>
> Seems very interesting. Am I wrong thinking this could helpto create
> multi-lingual glossary as drafted in
> https://phabricator.wikimedia.org/T150263#2860014 ?
>
>
> Le 22/12/2016 à 20:30, Yuri Astrakhan a écrit :
> > Gift season! We have launched structured data on Commons, available from
> > all wikis.
> >
> > TLDR; One data store. Use everywhere. Upload table data to Commons, with
> > localization, and use it to create wiki tables, lists, or use directly in
> > graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this
> > per-state GDP map demo, and select multiple years. More demos at the
> bottom.
> > US Map state highlight
> > <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight>
> >
> > Data can now be stored as *.tab and *.map pages in the data namespace on
> > Commons. That data may contain localization, so a table cell could be in
> > multiple languages. And that data is accessible from any wikis, by Lua
> > scripts, Graphs, and Maps.
> >
> > Lua lets you generate wiki tables from the data by filtering, converting,
> > mixing, and formatting the raw data. Lua also lets you generate lists. Or
> > any wiki markup.
> >
> > Graphs can use both .tab and .map directly to visualize the data and let
> > users interact with it. The GDP demo above uses a map from Commons, and
> > colors each segment with the data based on a data table.
> >
> > Kartographer (/) can use the .map data as an extra
> layer
> > on top of the base map. This way we can show endangered species' habitat.
> >
> > == Demo ==
> > * Raw data example
> > <https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab>
> > * Interactive Weather data
> > <https://en.wikipedia.org/wiki/Template:Graph:Weather_monthly_history>
> > * Same data in Weather template
> > <https://en.wikipedia.org/wiki/User:Yurik/WeatherDemo>
> > * Interactive GDP map
> > <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight>
> > * Endangered Jemez Mountains salamander - habitat
> > <https://en.wikipedia.org/wiki/Jemez_Mountains_salamander#/maplink/0>
> > * Population history
> > <https://en.wikipedia.org/wiki/Template:Graph:Population_history>
> > * Line chart <https://en.wikipedia.org/wiki/Template:Graph:Lines>
> >
> > == Getting started ==
> > * Try creating a page at data:Sandbox/.tab on Commons. Don't forget
> > the .tab extension, or it won't work.
> > * Try using some data with the Line chart graph template
> > A thorough guide is needed, help is welcome!
> >
> > == Documentation links ==
> > * Tabular help <https://www.mediawiki.org/wiki/Help:Tabular_Data>
> > * Map help <https://www.mediawiki.org/wiki/Help:Map_Data>
> > If you find a bug, create Phabricator ticket with #tabular-data tag, or
> > comment on the documentation talk pages.
> >
> > == FAQ ==
> > * Relation to Wikidata:  Wikidata is about "facts" (small pieces of
> > information). Structured data is about "blobs" - large amounts of data
> like
> > the historical weather or the outline of the state of New York.
> >
> > == TODOs ==
> > * Add a nice "table editor" - editing JSON by hand is cruel. T134618
> > * "What links here" should track data usage across wikis. Will allow
> > quicker auto-refresh of the pages too. T153966
> > * Support data redirects. T153598
> > * Mega epic: Support external data feeds.
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Now live: Shared structured data

[Yuri Astrakhan]

The 400 chat limit is to be in sync with Wikidata, which has the same
limitation. The origins of this limit is to encourage storage of "values"
rather than full strings (sentences). Also, it discourages storage of wiki
markup.

On Wed, Dec 28, 2016, 16:45 mathieu stumpf guntz <
psychosl...@culture-libre.org> wrote:

> Thank you Yuri. Is there some rational explanation behind this limits? I
> understand the limit over performance concern, and 2Mb seems already
> very large for intented glossaries. But 400 chars might be problematic
> for some definition I guess, especially since translations can lead to
> varying lenght needs.
>
>
> Le 25/12/2016 à 17:03, Yuri Astrakhan a écrit :
> > Hi Mathieu, yes, I think you can totally build up this glossary in a
> > dataset. Just remember that each string can be no longer then 400 chars,
> > and total size under 2mb.
> >
> > On Sun, Dec 25, 2016, 10:45 mathieu stumpf guntz <
> > psychosl...@culture-libre.org> wrote:
> >
> >> Hi Yuri,
> >>
> >> Seems very interesting. Am I wrong thinking this could helpto create
> >> multi-lingual glossary as drafted in
> >> https://phabricator.wikimedia.org/T150263#2860014 ?
> >>
> >>
> >> Le 22/12/2016 à 20:30, Yuri Astrakhan a écrit :
> >>> Gift season! We have launched structured data on Commons, available
> from
> >>> all wikis.
> >>>
> >>> TLDR; One data store. Use everywhere. Upload table data to Commons,
> with
> >>> localization, and use it to create wiki tables, lists, or use directly
> in
> >>> graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this
> >>> per-state GDP map demo, and select multiple years. More demos at the
> >> bottom.
> >>> US Map state highlight
> >>> <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight>
> >>>
> >>> Data can now be stored as *.tab and *.map pages in the data namespace
> on
> >>> Commons. That data may contain localization, so a table cell could be
> in
> >>> multiple languages. And that data is accessible from any wikis, by Lua
> >>> scripts, Graphs, and Maps.
> >>>
> >>> Lua lets you generate wiki tables from the data by filtering,
> converting,
> >>> mixing, and formatting the raw data. Lua also lets you generate lists.
> Or
> >>> any wiki markup.
> >>>
> >>> Graphs can use both .tab and .map directly to visualize the data and
> let
> >>> users interact with it. The GDP demo above uses a map from Commons, and
> >>> colors each segment with the data based on a data table.
> >>>
> >>> Kartographer (/) can use the .map data as an extra
> >> layer
> >>> on top of the base map. This way we can show endangered species'
> habitat.
> >>>
> >>> == Demo ==
> >>> * Raw data example
> >>> <https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab>
> >>> * Interactive Weather data
> >>> <https://en.wikipedia.org/wiki/Template:Graph:Weather_monthly_history>
> >>> * Same data in Weather template
> >>> <https://en.wikipedia.org/wiki/User:Yurik/WeatherDemo>
> >>> * Interactive GDP map
> >>> <https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight>
> >>> * Endangered Jemez Mountains salamander - habitat
> >>> <https://en.wikipedia.org/wiki/Jemez_Mountains_salamander#/maplink/0>
> >>> * Population history
> >>> <https://en.wikipedia.org/wiki/Template:Graph:Population_history>
> >>> * Line chart <https://en.wikipedia.org/wiki/Template:Graph:Lines>
> >>>
> >>> == Getting started ==
> >>> * Try creating a page at data:Sandbox/.tab on Commons. Don't
> forget
> >>> the .tab extension, or it won't work.
> >>> * Try using some data with the Line chart graph template
> >>> A thorough guide is needed, help is welcome!
> >>>
> >>> == Documentation links ==
> >>> * Tabular help <https://www.mediawiki.org/wiki/Help:Tabular_Data>
> >>> * Map help <https://www.mediawiki.org/wiki/Help:Map_Data>
> >>> If you find a bug, create Phabricator ticket with #tabular-data tag, or
> >>> comment on the documentation talk pages.
> >>>
> >>> == FAQ ==
> >>> * Relation to Wikidata:  Wikidata is about "facts" (small pieces of
> 

[Wikitech-l] Localizable cross-wiki templates

[Yuri Astrakhan]

TLDR: Templates can now use datasets on Commons for all their localizable
messages, making it possible to copy/paste templates between wiki without
any changes.

If you look at the Graph:Lines template [1], there is a small link under
each graph that points to the data source. That link is automatically
localized on enwiki and ruwiki because the actual messages are stored in a
Commons' Dataset, while the template itself is identical on both wikis.

The magic happens in the Module:TNT (hoping it won't blow up):

*{{#invoke:TNT | msg | DatasetDict.tab | message-key | param1 | param2 |
... }}*

I would love to support template doc pages too, but I'm not sure how well
VE and other tools will work with the 

This is an experimental thing, just hacking on it in my free time, and
would love to get your feedback on the viability of this approach. Thanks!

Links:
enwiki: https://en.wikipedia.org/wiki/Template:Graph:Lines
ruwiki: https://ru.wikipedia.org/wiki/Template:Graph:Lines
TNT: https://commons.wikimedia.org/wiki/Module:TNT
Dictionary:
https://commons.wikimedia.org/wiki/Data:Original/Template:Graph:Lines.tab
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Map replaces GeoHack on ruwiki

[Yuri Astrakhan]

Russian Wikipedia just replaced all of their map links in the upper right
corner (geohack) with the  - Kartographer extension!  Moreover,
when clicking the link, it also shows the location outline, if that object
exists in OpenStreetMap, using corresponding Wikidata ID.  My deepest
respect to my former Interactive Team colleagues and volunteers who have
made it possible!  (This was community wishlist #21)

Example - city of Salzburg (click coordinates in the upper right corner, or
in the infobox):
https://ru.wikipedia.org/wiki/%D0%97%D0%B0%D0%BB%D1%8C%D1%86%D0%B1%D1%83%D1%80%D0%B3
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Yuri Astrakhan]

Thanks Legoktm!

For the reference, I was granted these rights as a volunteer, before
joining WMF: https://lists.gt.net/wiki/wikitech/335950

On Tue, Jan 24, 2017 at 10:44 PM, Legoktm 
wrote:

> Hi,
>
> After speaking with Yurik, I've filed
>  on his behalf to restore his
> membership in the mediawiki and maps-dev groups.
>
> I would appreciate guidance in whether these rights can be summarily
> granted since he used to have them, or if it needs to go through the
> full process.
>
> -- Legoktm
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] +2 request for yurik in mediawiki and maps-dev

[Yuri Astrakhan]

How is that different from becoming disgruntled while still working at WMF?
Or becoming disgruntled without ever joining? Or stolen credentials?

On Wed, Jan 25, 2017, 16:01 Kevin Smith  wrote:

On Wed, Jan 25, 2017 at 12:55 PM, Alex Monk  wrote:

> You don't appear to be a developer. Even if you were, Brian said 'similar
> situation' - i.e., where someone had been granted rights before becoming
> WMF staff.
>

​I am a developer, but not a wikimedia developer (yet?). That's why I'm not
exactly clear on the potential risks of giving +2 rights to someone who
intends harm.

We are talking about exactly the same situation: Someone was a productive
volunteer, then staff, then no longer staff. 99+% of the time, they should
retain their rights, or get them back shortly after leaving. But there may
be cases where someone would be disgruntled, and if rights are automatic in
all cases, such a person could do damage. I'm just suggesting that there
should be a quick and simple check to be sure that's not the case. Being
without rights for a couple days doesn't seem like a small price to pay for
the safety of the project.

​Kevin​
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Multilingual maps: how to pick each feature language?

[Yuri Astrakhan]

TLDR: if browsing a map for French wiki, and a city only has a Russian and
Chinese name, which one should be shown? Should the city name have
different rules from a store or a street name? ...

I have been hacking to add unlimited multilingual support to Wikipedia
maps, and have language fallback question:  given a list of arbitrary
languages for each map feature, what is the best choice for a given
language?

I know Mediawiki has language fallbacks, but they are very simple (e.g. for
"ru", if "ru" is not there, try "en").

Some things to consider:
* Unlike Wikipedia, where readers go for "meaning", in maps we mostly need
"readability".
* Alphabets: Latin alphabet is probably the most universally understood,
followed by...? Per target language?
* Politics: places like Crimea tend to have both Russian and Ukrainian
names defined, but if drawing map in Ukrainian, and some feature has
Russian and English names, but not Ukrainian, should it be shown with the
Russian or Ukrainian name?
* When viewing a map of China in English, should Chinese (local) name be
shown together with the English name? Should it be shown for all types of
features (city name, street name, name of the church, ...?)

Thanks!
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l