[Wikitech-l] Implementing features for the sake of implementing features

[Eran Rosenthal]

Sometimes programmers waste their time for implementing
non-useful/non-important features, but sometimes such features aren't just
not useful, but harmful.[1; you are more than welcome to convince me
otherwise]

Both developers, designers and project managers should always ask
themselves why do we need such feature before going to implement it and
should be able to convince others on the motivation for it.

I think that sometimes we fail… (and sometimes even in the "postmortem"
step[1]) The question is how can Wikimedia foundation improve the process?

For  large changes there are already reviews and RFC on mediawiki wiki, but
for smaller ones it is sometimes missed (and even phabricator ticket).
Should any feature should be associated with phab task?[2]


Eranroz

-

[1]  https
://
phabricator.wikimedia.org
/T100691
 /
https://gerrit.wikimedia.org/r/#/c/95723/ defeature __NOEDITSECTION__ from
VE

[2]
https://www.mediawiki.org/wiki/VisualEditor/2015_Process_Review#Recommendation:_Maintain_Goals.2C_Epics.2C_and_Tasks_in_alignment_in_Phabricator
.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] API BREAKING CHANGE: Default continuation mode for action=query will change at the end of this month

[Yuri Astrakhan]

On Wed, Jun 17, 2015 at 7:44 PM, John Mark Vandenberg 
wrote:

>
> The API currently emits a warning if a query continuation mode isnt
> selected.
>
> I guess on July 1 the API could emit an error, and not return any query
> data.
> Then the data isnt going to cause weird behaviour - it will break,
> properly.
>
>
Hmm, this is actually an interesting idea - would it make sense to error on
missing "continue" or "rawcontinue" for all action=query for about a month
or two, so that everyone notices it right away and gets updated, and than
resume with the new behavior?
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] API BREAKING CHANGE: Default continuation mode for action=query will change at the end of this month

[John Mark Vandenberg]

On Wed, Jun 17, 2015 at 9:15 PM, Marco  wrote:
> On Wed, 03 Jun 2015 18:29:08 +0700, John Mark Vandenberg wrote:
>
>> On Wed, Jun 3, 2015 at 3:42 AM, Brad Jorsch (Anomie)
>>  wrote:
>>> ...
>>> I've compiled a list of bots that have hit the deprecation warning more
>>> than 1 times over the course of the week May 23–29. If you are
>>> responsible for any of these bots, please fix them. If you know who is,
>>> please make sure they've seen this notification. Thanks.
>>
>> Thank you Brad for doing impact analysis and providing a list of the 71
>> bots with more than 10,000 problems per week.  We can try to solve those
>> by working with the bot operators.
>>
>> If possible, could you compile a list of bots affected at a lower
>> threshold - maybe 1,000.  That will give us a better idea of the scale
>> of bots operators that will be affected when this lands - currently in
>> one months time.
>>
>> Will the deploy date be moved back if the impact doesnt diminish by bots
>> being fixed?
>
> Should someone contact those bots on their talk page to notify the owners or
> do we hope everyone reads this mailing list?

I see Whatamidoing (WMF) has already done this for many bots.

I saw one of the problematic bots says they are using AutoWikiBrowser.
maybe the bot is using an old version of AWB?

> Also, this change will affect not only bots but also every piece of software
> or tool that was written or published prior to the change and relies on the
> API to fetch data.
> So there are two issues with that:
> * The maintainer of the tool has no time/interest to compile and publish new
> releases of versions which were considered stable. This means the tool will
> die if the source code is not available or no one wants to take over.
> * Previously published programs that were considered stable and are executed
> after July 1st. The user may not be aware that this tool is no longer stable
> and requires an update. As far as I understood this API change: The results
> from the API will just be somewhat different and not produce an error. So
> the software will not crash but may produce weird behavior instead. Is there
> any solution to this?

The API currently emits a warning if a query continuation mode isnt selected.

I guess on July 1 the API could emit an error, and not return any query data.
Then the data isnt going to cause weird behaviour - it will break, properly.

-- 
John Vandenberg

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Overriding Jenkins versus make-work patches

[Legoktm]

On 06/17/2015 09:53 AM, Brad Jorsch (Anomie) wrote:
>1. Merge the core change over Jenkins's objections, then the Flow change
>can be merged as normal. But overriding Jenkins sucks.

Force-merging in a jenkins/zuul managed repository should be avoided as
much as possible, since it will cause zuul to deadlock and freeze[1].

>2. Split the core patch into two parts: part 1 does everything except
>add the wfDeprecated() call, while part 2 adds just the wfDeprecated() call
>and will be merged immediately after. The make-work here just to make
>Jenkins happy sucks and slightly clutters the commit history.

I would prefer this one.


[1] https://phabricator.wikimedia.org/T93812

-- Legoktm

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] 2015-06-17 Scrum of Scrums notes

[Grace Gellerman]

https://www.mediawiki.org/wiki/Scrum_of_scrums/2015-06-17
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Overriding Jenkins versus make-work patches

[S Page]

#1 sounds right, Jenkins serves us, not vice versa. The core change's
commit message should reference the two required extension changes.
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Overriding Jenkins versus make-work patches

[Bryan Davis]

On Wed, Jun 17, 2015 at 10:53 AM, Brad Jorsch (Anomie)
 wrote:
> We have an (informal?) policy that deprecation warnings shouldn't be raised
> in WMF production. Thus if a core patch is going to add deprecation
> warnings we have to make sure that all extensions deployed on the cluster
> are updated to not trigger the warning. We can accomplish this by carefully
> managing the addition of the warnings, by detecting the core version from
> the extension and changing behavior, or by simply updating both core and
> extension at the same time.
>
> With Gerrit change 134827,[1] the affected extensions are Flow and
> CentralAuth. For neither of these extensions do we care that
> extension-master works with non-master versions of MediaWiki core, and
> patches to update the extensions are ready.[2][3] So normally we'd just
> merge all three at once and be done with it.
>
> The problem is unit tests: we can't merge the core change[1] first because
> Flow's unit tests will fail on the deprecation warning, and we can't merge
> the Flow change[2] first because it doesn't work without the core change.
> There are various ways to work around this, but all are ugly:
>
>1. Merge the core change over Jenkins's objections, then the Flow change
>can be merged as normal. But overriding Jenkins sucks.
>2. Split the core patch into two parts: part 1 does everything except
>add the wfDeprecated() call, while part 2 adds just the wfDeprecated() call
>and will be merged immediately after. The make-work here just to make
>Jenkins happy sucks and slightly clutters the commit history.
>3. Rewrite the Flow unit test to detect whether core has the core change
>and alter behavior accordingly. This is even more make-work than option 2
>when we're otherwise happy to just coordinate the merges.
>
> Which ugly option do we as a development community prefer in this
> situation? Personally I'd go for option 1 as the most expedient with the
> fewest long-term consequences.
>
>  [1]: https://gerrit.wikimedia.org/r/#/c/134827/
>  [2]: https://gerrit.wikimedia.org/r/#/c/190023/
>  [3]: https://gerrit.wikimedia.org/r/#/c/190026/
>
>
> P.S. Let's not side-track this into whether the "extension master only
> needs to be compatible with core master" policy for Flow and CentralAuth is
> good/bad, or that situations exist where options 2 or 3 are necessary
> choices (e.g. #2 when the extension fixes aren't ready yet and #3 when an
> extension involved doesn't have the "master is only compatible with core
> master" policy), or whether allowing core changes to be merged that cause
> non-WMF-deployed extensions to raise deprecation warnings is somehow
> discriminating against non-WMF-deployed extensions. Start a new thread if
> you want to discuss those, please.

For the similar but slightly different case of library version bumps
in composer.json and the associated mediawiki/vendor repo that holds
the Jenkins/Beta cluster/Prod realization of composer.json we use the
force past Jenkins option (#1). Both patches are put into Gerrit and
fail Jenkins tests: in core because functionality from the updated
library isn't available and in vendor because the library doesn't
match the version in core's composer.json. The vendor patch can be
forced and then the core patch retested to ensure that core is in the
right state with the updated vendor.

Bryan
-- 
Bryan Davis  Wikimedia Foundation
[[m:User:BDavis_(WMF)]]  Sr Software EngineerBoise, ID USA
irc: bd808v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Overriding Jenkins versus make-work patches

[Brad Jorsch (Anomie)]

We have an (informal?) policy that deprecation warnings shouldn't be raised
in WMF production. Thus if a core patch is going to add deprecation
warnings we have to make sure that all extensions deployed on the cluster
are updated to not trigger the warning. We can accomplish this by carefully
managing the addition of the warnings, by detecting the core version from
the extension and changing behavior, or by simply updating both core and
extension at the same time.

With Gerrit change 134827,[1] the affected extensions are Flow and
CentralAuth. For neither of these extensions do we care that
extension-master works with non-master versions of MediaWiki core, and
patches to update the extensions are ready.[2][3] So normally we'd just
merge all three at once and be done with it.

The problem is unit tests: we can't merge the core change[1] first because
Flow's unit tests will fail on the deprecation warning, and we can't merge
the Flow change[2] first because it doesn't work without the core change.
There are various ways to work around this, but all are ugly:

   1. Merge the core change over Jenkins's objections, then the Flow change
   can be merged as normal. But overriding Jenkins sucks.
   2. Split the core patch into two parts: part 1 does everything except
   add the wfDeprecated() call, while part 2 adds just the wfDeprecated() call
   and will be merged immediately after. The make-work here just to make
   Jenkins happy sucks and slightly clutters the commit history.
   3. Rewrite the Flow unit test to detect whether core has the core change
   and alter behavior accordingly. This is even more make-work than option 2
   when we're otherwise happy to just coordinate the merges.

Which ugly option do we as a development community prefer in this
situation? Personally I'd go for option 1 as the most expedient with the
fewest long-term consequences.

 [1]: https://gerrit.wikimedia.org/r/#/c/134827/
 [2]: https://gerrit.wikimedia.org/r/#/c/190023/
 [3]: https://gerrit.wikimedia.org/r/#/c/190026/


P.S. Let's not side-track this into whether the "extension master only
needs to be compatible with core master" policy for Flow and CentralAuth is
good/bad, or that situations exist where options 2 or 3 are necessary
choices (e.g. #2 when the extension fixes aren't ready yet and #3 when an
extension involved doesn't have the "master is only compatible with core
master" policy), or whether allowing core changes to be merged that cause
non-WMF-deployed extensions to raise deprecation warnings is somehow
discriminating against non-WMF-deployed extensions. Start a new thread if
you want to discuss those, please.



-- 
Brad Jorsch (Anomie)
Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] (off) Baidu Baike current dump

[Liangent]

I don't think there is and will be one, besides crawling web pages,
considering Baidu's past activities.

Note that contents on Baidu Baike are usually not free and in an
unknown / messy copyright status.

-Liangent

On Wed, Jun 17, 2015 at 8:20 PM, Farkas, Illes  wrote:
> Hello,
>
> Does anyone know a method for downloading the current dump of Baidu Baike?
> Suggestions for alternative mailing lists would be also great.
>
> Thanks,
> Illes
> http://goo.gl/trcz4
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] (off) Baidu Baike current dump

[Farkas, Illes]

Hello,

Does anyone know a method for downloading the current dump of Baidu Baike?
Suggestions for alternative mailing lists would be also great.

Thanks,
Illes
http://goo.gl/trcz4
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] API BREAKING CHANGE: Default continuation mode for action=query will change at the end of this month

[Marco]

On Wed, 03 Jun 2015 18:29:08 +0700, John Mark Vandenberg wrote:

> On Wed, Jun 3, 2015 at 3:42 AM, Brad Jorsch (Anomie)
>  wrote:
>> ...
>> I've compiled a list of bots that have hit the deprecation warning more
>> than 1 times over the course of the week May 23–29. If you are
>> responsible for any of these bots, please fix them. If you know who is,
>> please make sure they've seen this notification. Thanks.
> 
> Thank you Brad for doing impact analysis and providing a list of the 71
> bots with more than 10,000 problems per week.  We can try to solve those
> by working with the bot operators.
> 
> If possible, could you compile a list of bots affected at a lower
> threshold - maybe 1,000.  That will give us a better idea of the scale
> of bots operators that will be affected when this lands - currently in
> one months time.
> 
> Will the deploy date be moved back if the impact doesnt diminish by bots
> being fixed?

Should someone contact those bots on their talk page to notify the owners or
do we hope everyone reads this mailing list?

Also, this change will affect not only bots but also every piece of software
or tool that was written or published prior to the change and relies on the
API to fetch data. So there are two issues with that:
* The maintainer of the tool has no time/interest to compile and publish new
releases of versions which were considered stable. This means the tool will
die if the source code is not available or no one wants to take over.
* Previously published programs that were considered stable and are executed
after July 1st. The user may not be aware that this tool is no longer stable
and requires an update. As far as I understood this API change: The results
from the API will just be somewhat different and not produce an error. So
the software will not crash but may produce weird behavior instead. Is there
any solution to this?

Thanks,
Marco
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] RFC meeting this week

[Tim Starling]

In the next RFC meeting, we will discuss the following RFC:

* Request timeouts and retries
https://phabricator.wikimedia.org/T97204

* Re-evaluate varnish-level request-restart behavior on 5xx
https://phabricator.wikimedia.org/T97206

The meeting will be on the IRC channel #wikimedia-office on
chat.freenode.net at the following time:

* UTC: Wednesday 21:00
* US PDT: Wednesday 14:00
* Europe CEST: Wednesday 23:00
* Australia AEST: Thursday 07:00

-- Tim Starling


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l