[Wikitech-l] Phabricator monthly statistics - 2016-08

[communitymetrics]

Hi Community Metrics team,

This is your automatic monthly Phabricator statistics mail.

Accounts created in (2016-08): 299
Active users (any activity) in (2016-08): 892
Task authors in (2016-08): 502
Users who have closed tasks in (2016-08): 273

Projects which had at least one task moved from one column to another on
their workboard in (2016-08): 249

Tasks created in (2016-08): 2711
Tasks closed in (2016-08): 2442
Open and stalled tasks in total: 31103

Median age in days of open tasks by priority:

Unbreak now: 67
Needs Triage: 194
High: 346
Normal: 497
Low: 779
Lowest: 640

(How long tasks have been open, not how long they have had that priority)

TODO: Numbers which refer to closed tasks might not be correct, as
described in https://phabricator.wikimedia.org/T1003 .

Yours sincerely,
Fab Rick Aytor

(via community_metrics.sh on iridium at Thu Sep  1 00:00:13 UTC 2016)

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] to be enabled tomorrow

[James Forrester]

Great to see. Thank you for all your work on this, Discovery!

On Wed, 31 Aug 2016 at 14:16 Yuri Astrakhan 
wrote:

> Seems we have no more blockers for  tag on all Wikipedias. We will
> enable it tomorrow, Sept 1st.  tag allows editors to add a link to
> a popup map, complete with extra geojson overlay data.
>
> See help page for instructions:
> https://www.mediawiki.org/wiki/Help:Extension:Kartographer#.3Cmaplink.3E
>
> Submit bugs to phabricator:
>
> https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=Kartographer,maps
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

-- 

James D. Forrester
Lead Product Manager, Editing
Wikimedia Foundation, Inc.
jforrester at wikimedia.org
 |
@jdforrester
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] to be enabled tomorrow

[Yuri Astrakhan]

Seems we have no more blockers for  tag on all Wikipedias. We will
enable it tomorrow, Sept 1st.  tag allows editors to add a link to
a popup map, complete with extra geojson overlay data.

See help page for instructions:
https://www.mediawiki.org/wiki/Help:Extension:Kartographer#.3Cmaplink.3E

Submit bugs to phabricator:
https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?tags=Kartographer,maps
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] 2016W35 ArchCom-RFC meeting: Revisiting image/oldimage (T589)

[Rob Lanphier]

My apologies.  I made a confusing thinko in the announcement about the
ArchCom office hour in 2.5 hours.  I wrote:
"schema change for page content language" (last week's RFC discussion)

...when I meant:
"image and oldimage tables" (what we'd like to revisit this week)

Corrected version below (highlighted with "***")

Rob

On Wed, Aug 31, 2016 at 12:20 AM, Rob Lanphier  wrote:
> Hi everyone,
>
> For this week's office hour, we'd like to discuss [T589: ***RfC: image and 
> oldimage tables***][1]  Timo brought this up on the list a
> couple of weeks ago, and didn't get much of a response. (But thank you
> Jaime for responding!)  We discussed this one fairly recently ([July
> 13][2]).  This area of our system is still in need of simplification
> and optimization.
>
> The discussion is scheduled for 2016-08-31 UTC:
> Time: Wednesday 21 UTC (2pm PDT, 23 CEST)
> Place: #wikimedia-office
> Phab event: [E266][3]
> [ArchCom/Status][4]
>
> Rob
>
> [1]: 
> [3]:  July 13 meeting
> [3]:  Upcoming meeting
> [4]: 
> -- Forwarded message --
> From: Krinkle 
> Date: Wed, Aug 10, 2016 at 1:54 PM
> Subject: [Wikitech-l] Schema migration for 'image' and 'oldimage' tables
> To: Wikimedia developers 
>
>
> TL;DR: Participate on T589 and help decide what the upcoming schema change
> should entail, and how we'll migrate existing data.
>
> Hey all,
>
> Couple weeks ago we dedicated an IRC office hour to
> https://phabricator.wikimedia.org/T589 (RFC: image and oldimage tables).
>
> Updated draft at:
> https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tables
>
> We clarified scope and purpose of this particular RFC. Other issues are
> still important but considered orthogonal, and to be dealt with parallelly
> (or at a later time).
>
> Revised problem statement:
>
> 1. File revisions should have unique identifiers (better than "current file
> title + upload timestamp". (Subject to race conditions, hard to
> index/query, etc.)
> 2. Uploading new file revisions must not involve rows moving across tables,
> or rows being replaced.
>
> Participants agreed with the revised problem statement, it makes sense not
> to merely add primary keys to the existing tables ("Proposal 1" on the RFC
> draft), as that wouldn't adequately solve the Problem 2.
>
> The second proposal was to separate information about image revision from
> the image entity itself. Similar to the page/revisions tables. This was
> generally accepted as a good idea, but details are still to be determined.
>
> The general idea is that all revision-specific information (except for a
> pointer to the current revision) would no longer live in the 'image' table.
> Instead, information about all (for both current and past revisions) would
> live in the same table (instead of being moved around from one table to
> another when it's no longer the current one).
>
> Details at:
> https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tables#2._Separate_entity_and_versioning
>
> Some open questions I'd like to see discussed on Phabricator (or here on
> wikitech):
>
> 1. Which fields do we keep in the 'image' table (img_id, img_name,
> img_latest, anything else?).
>
> All fields currently being queried from both tables, will probably only
> stay in the image revision table. But there are a few fields that we
> intentionally only want to query about current versions. For example
> 'img_sha1'. For duplicate-detection, we need to only consider the latest
> revisions. Doing this by keeping img_sha1 means uploading a new revision
> will involve updating two fields instead of one (img_latest and img_sha1).
> This isn't unprecedented as we do this for page as well
> (WikiPage::updateRevisionOn; page_latest, page_touched, page_is_redirect,
> page_len).
>
> Are there other fields we need to keep besides img_sha1? Or should we can
> solve the img_sha1 use case in a different manner?
>
> 2. img_metadata
>
> This field is a blob of serialised PHP (typically representing the Exif
> data of an image).
>
> Tim (correct me if I got it wrong) mentioned we could potentially make
> migration easier by changing img_metadata to be stored in a separate table
> and change the img_metadata field (in the image revision table) to instead
> be a pointer to a primary key.
>
> This could potentially be done separately later, but if it helps migration,
> we should consider doing it now.
>
> How will this interact with file deletion? Will it be difficult to garbage
> collect this? Do we need to? (We don't seem to do it for the 'text' table /
> external store; is it worth moving this an external store?)
>
> 3. Migration
>
> If we rename both tables (image/oldimage -> file/filerevision), we'd have
> the ability to run migration in the background without interfering with the
> live site, 

[Wikitech-l] 2016-08-31 Scrum of Scrums meeting notes

[Grace Gellerman]

https://www.mediawiki.org/wiki/Scrum_of_scrums/2016-08-31


== Product ==

=== Reading ===

 Reading Web 
* Current sprint:
* Fixing lazy loaded images bugs (ex: mathml formulas)
* Diagnosed problem with hovercards EL data. Will submit fix next sprint
* Train blocked and unblocked on MediawikiServices introduction on
MobileFrontend
* Next sprint:
* Carry on work on footer work and lazy loaded images work
* Related pages improvements
* Shipping wikidata descriptions in mobile web to some wikis

  Mobile Content Service (MCS) 
* Plus symbol in title fix work in progress
** blocked on issue with url fragments (e.g. /wiki/[title]#[section])
* Trending service standup ongoing, some field renaming work, probably
better to move from rcstream to ChangeProp
* Likely 'On this day' service work to start in next several weeks

 Android native app 
* Current sprint: https://phabricator.wikimedia.org/project/view/2178/
** continuing navigation overhaul; forecasting to have it complete this
sprint.
** made an 'interim' release to production, with some Feed features that
were most requested by users.
 * Next sprint:
 * planning to complete design touch-ups and get ready to release.


 iOS native app 
* Current release board:
https://phabricator.wikimedia.org/project/board/1736/
** 5.1 set to be released today or tomorrow
** We found a late minor regression affecting citation links, but likely
* Next board (no change):
https://phabricator.wikimedia.org/project/view/2150/
** 5.2 is in developmentwith expected deployment alongside iOS 10 release
in late September
*** Adding iOS 10 support (with widgets)
*** Dropping iOS 8 support


   -

 Reading Infrastructure 
* nothing blocking/blocked

=== Community Tech ===
* Currently rolling out numeric collation to English Wikipedia (will take a
few more days for the script to complete)
* Rolling out PageAssessments to English Wikivoyage this week (possibly
English Wikipedia next week). Jamie will help us monitor.
* No blockers

=== Editing ===
 Collaboration 
* '''Blocking''':
** Continuing work on Flow caching rewrite for multi-DC.  We're now 1)
using WanCache, 2) deleting on write and setting cache on read.  Still
verifying that everything is working properly.
* '''Blocked'':
* '''Updates''':
** Finished the work to unwatch from Echo notifications.
** Flow VE fixes
** Added a server-side message poster.  This is a way to post to a talk
page without knowing whether it uses Flow or wikitext.  We already have
this on the client as well.
** Issues with mw.notify.  We've temporarily re-implemented locally, but
want to resolve the core issues and use that.  See
https://gerrit.wikimedia.org/r/#/c/306560/ .

 Language 
* Blocking:
* Blocked:
* Updates:
** Apertium packaging work finished (Except kaz/kaz-tat), Kartik/Alex to
start work on Jessie migration for service.
** New CXStats page:
https://test.wikipedia.org/wiki/Special:ContentTranslationStats
** Work related to template adaption continue.
** MLEB released last week.

 Parsing 
* Blocked on security review of Parser Migration extension (I see now that
Security is on it)
* Ongoing work to clean up parser tests infrastructure
* Resumed work on Language Variants support in Parsoid (initial work to
attain rendering parity with PHP Parser output)
* Ongoing work with Linker rewrite as part of cleanup for the shadow
namespaces work

== Analytics ==

* loading of new AQS (pageview API) cluster ongoing, will switch over to it
when done, scaling and load testing docs avilable here:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Scaling
* new event bus logging from mw hooks merged, will start being available on
event bus soon
* browser dashboards with loads of traction as of late thanks to twitter
and blogpost: https://blog.wikimedia.org/2016/08/19/most-popular-browser/ (we
reached 2000 unqiue visits):
https://analytics.wikimedia.org/dashboards/browsers/#all-sites-by-os
* Currently evaluating druid/clickhouse as viable datastore for edit data



=== Security ===
* Security reviews this week:
** Youdao MT
** Catching up on overdue reviews -- comments for
https://phabricator.wikimedia.org/T141591 (HTML5Depurate and
ParserMigration will be posted later today)
* Darian working on hiring tasks
* Darian out next week (Sept. 6-9)
** Substitute with Phabricator understanding needed for Data Breach
Training on Sept. 9th at 11:00a.m. Pacific; just have to answer questions
that may come up with regard to setting Phab tickets private

=== Services ===
* Blocking: none
* Blocked: none
* Updates
** Parsoid move to scap3 completed
** Change-Prop is updating summary on wikidata item change now
** Summary endpoint includes wikidata description now
** New sections transform API to be deployed today

=== Technical Operations ===
* '''Blocking'''
** None
* '''Blocked'''
** HTML (RESTBase) dumps: script does not account for deleted
pages/content. - blocked by services -

Re: [Wikitech-l] MediaWiki 1.28 questions (isBot() and database)

[John]

db changes are fairly easy to take care of during an update (Just running
update.php handles all of it) I personally prefer the LTS branches due to
having less maintenance needs.

On Wed, Aug 31, 2016 at 11:08 AM, Bináris  wrote:

> https://phabricator.wikimedia.org/diffusion/MW/browse/
> master/RELEASE-NOTES-1.28
>
> * User::isBot() method for checking if an account is a bot role account.
> > * Added a new hook, 'UserIsBot', to aid in determining if a user is a
> bot.
> >
> Sounds great! We have been waiting for it for a long time.
> Will that work with a time parameter? Can I know somehow that the user WAS
> a bot in the time of an edit?
>
> Other question: release notes say, "1.28 has several database changes since
> 1.27, and will not work without schema
> updates."
> If I want to install a private wiki for private purposes on my own laptop
> (on user, offline work, no access from outside), which is better choice:
> stable version or 1.28 alpha to avoid database changes later?
>
> --
> Bináris
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] MediaWiki 1.28 questions (isBot() and database)

[Bináris]

https://phabricator.wikimedia.org/diffusion/MW/browse/master/RELEASE-NOTES-1.28

* User::isBot() method for checking if an account is a bot role account.
> * Added a new hook, 'UserIsBot', to aid in determining if a user is a bot.
>
Sounds great! We have been waiting for it for a long time.
Will that work with a time parameter? Can I know somehow that the user WAS
a bot in the time of an edit?

Other question: release notes say, "1.28 has several database changes since
1.27, and will not work without schema
updates."
If I want to install a private wiki for private purposes on my own laptop
(on user, offline work, no access from outside), which is better choice:
stable version or 1.28 alpha to avoid database changes later?

-- 
Bináris
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding namespaces

[Bináris]

2016-08-31 14:14 GMT+02:00 Petr Kadlec :

> https://hu.wikipedia.org/w/api.php?action=query&meta=
> siteinfo&siprop=namespaces|namespacealiases
>

Thank you!


-- 
Bináris
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding namespaces

[Petr Kadlec]

Hi,

On Wed, Aug 31, 2016 at 2:10 PM, Bináris  wrote:

> > > Additional question: where can I see available aliases as a user?
> > >
> >
> > https://meta.wikimedia.org/w/api.php?action=query&meta=siteinfo&siprop=
> > namespacealiases
>
> That's useful, but not complete again. Are there somewhere default aliases?
>

https://hu.wikipedia.org/w/api.php?action=query&meta=siteinfo&siprop=namespaces|namespacealiases

HTH,
-- [[cs:User:Mormegil | Petr Kadlec]]
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding namespaces

[Bináris]

2016-08-31 13:37 GMT+02:00 Alex Monk :

> On 31 August 2016 at 12:18, Bináris  wrote:
>
> > How can I find out from witihn a wiki, what namespaces itt uses?
> >
> Special:AllPages has a dropdown with each namespace.
>
That is half a solution, does not provide numbers.
I wonder how help and help talk is there, once we have made it removed from
huwiki.

>
> On 31 August 2016 at 12:18, Bináris  wrote:
>
> > Additional question: where can I see available aliases as a user?
> >
>
> https://meta.wikimedia.org/w/api.php?action=query&meta=siteinfo&siprop=
> namespacealiases

That's useful, but not complete again. Are there somewhere default aliases?
Thank you!
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Finding namespaces

[Alex Monk]

On 31 August 2016 at 12:18, Bináris  wrote:

> How can I find out from witihn a wiki, what namespaces itt uses?
>
Special:AllPages has a dropdown with each namespace.

On 31 August 2016 at 12:18, Bináris  wrote:

> Additional question: where can I see available aliases as a user?
>

https://meta.wikimedia.org/w/api.php?action=query&meta=siteinfo&siprop=namespacealiases
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Finding namespaces

[Bináris]

Hi,

How can I find out from witihn a wiki, what namespaces itt uses? I thought,
Special:Version would be a place to find, but no.
(I mined it from Pywikibot at last.)
Additional question: where can I see available aliases as a user?

-- 
Bináris
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Phab blogs

[Quim Gil]

Hi,

On Wed, Aug 31, 2016 at 8:03 AM, Tilman Bayer  wrote:

> Apparently there hasn't been much discussion or deliberation about the
> advantages and disadvantages of this new feature compared to other
> communication channels.


Excess of communication channels is a big problem in Wikimedia and a
regular topic for complaints from new and experienced contributors
(volunteers or paid, tech or non-tech). I think such discussion about
advantages and disadvantages should happen before we invest too much
content and attention in Phabricator blogs.

-- 
Quim Gil
Engineering Community Manager @ Wikimedia Foundation
http://www.mediawiki.org/wiki/User:Qgil
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 2016W35 ArchCom-RFC meeting: Revisiting image/oldimage (T589)

[Rob Lanphier]

Hi everyone,

For this week's office hour, we'd like to discuss [T589: schema change
for page content language][1]  Timo brought this up on the list a
couple of weeks ago, and didn't get much of a response. (But thank you
Jaime for responding!)  We discussed this one fairly recently ([July
13][2]).  This area of our system is still in need of simplification
and optimization.

The discussion is scheduled for 2016-08-31 UTC:
Time: Wednesday 21 UTC (2pm PDT, 23 CEST)
Place: #wikimedia-office
Phab event: [E266][3]
[ArchCom/Status][4]

Rob

[1]: 
[3]:  July 13 meeting
[3]:  Upcoming meeting
[4]: 
-- Forwarded message --
From: Krinkle 
Date: Wed, Aug 10, 2016 at 1:54 PM
Subject: [Wikitech-l] Schema migration for 'image' and 'oldimage' tables
To: Wikimedia developers 


TL;DR: Participate on T589 and help decide what the upcoming schema change
should entail, and how we'll migrate existing data.

Hey all,

Couple weeks ago we dedicated an IRC office hour to
https://phabricator.wikimedia.org/T589 (RFC: image and oldimage tables).

Updated draft at:
https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tables

We clarified scope and purpose of this particular RFC. Other issues are
still important but considered orthogonal, and to be dealt with parallelly
(or at a later time).

Revised problem statement:

1. File revisions should have unique identifiers (better than "current file
title + upload timestamp". (Subject to race conditions, hard to
index/query, etc.)
2. Uploading new file revisions must not involve rows moving across tables,
or rows being replaced.

Participants agreed with the revised problem statement, it makes sense not
to merely add primary keys to the existing tables ("Proposal 1" on the RFC
draft), as that wouldn't adequately solve the Problem 2.

The second proposal was to separate information about image revision from
the image entity itself. Similar to the page/revisions tables. This was
generally accepted as a good idea, but details are still to be determined.

The general idea is that all revision-specific information (except for a
pointer to the current revision) would no longer live in the 'image' table.
Instead, information about all (for both current and past revisions) would
live in the same table (instead of being moved around from one table to
another when it's no longer the current one).

Details at:
https://www.mediawiki.org/wiki/Requests_for_comment/image_and_oldimage_tables#2._Separate_entity_and_versioning

Some open questions I'd like to see discussed on Phabricator (or here on
wikitech):

1. Which fields do we keep in the 'image' table (img_id, img_name,
img_latest, anything else?).

All fields currently being queried from both tables, will probably only
stay in the image revision table. But there are a few fields that we
intentionally only want to query about current versions. For example
'img_sha1'. For duplicate-detection, we need to only consider the latest
revisions. Doing this by keeping img_sha1 means uploading a new revision
will involve updating two fields instead of one (img_latest and img_sha1).
This isn't unprecedented as we do this for page as well
(WikiPage::updateRevisionOn; page_latest, page_touched, page_is_redirect,
page_len).

Are there other fields we need to keep besides img_sha1? Or should we can
solve the img_sha1 use case in a different manner?

2. img_metadata

This field is a blob of serialised PHP (typically representing the Exif
data of an image).

Tim (correct me if I got it wrong) mentioned we could potentially make
migration easier by changing img_metadata to be stored in a separate table
and change the img_metadata field (in the image revision table) to instead
be a pointer to a primary key.

This could potentially be done separately later, but if it helps migration,
we should consider doing it now.

How will this interact with file deletion? Will it be difficult to garbage
collect this? Do we need to? (We don't seem to do it for the 'text' table /
external store; is it worth moving this an external store?)

3. Migration

If we rename both tables (image/oldimage -> file/filerevision), we'd have
the ability to run migration in the background without interfering with the
live site, and without requiring a long read-only period and/or duplicate
and additional code complexity to be developed.

Is there a way we can do the migration without creating two new tables?
Using the oldimage table as import destination for current rows isn't
straight forward as existing scan queries would need to skip the current
rows somehow while in the midst of this migration. Seems possible, but is
it worth the complexity? (We'd need extra code that knows about that
migration field, and how long do we keep that code? Also complicates
migration for third-parties usi