It would be nice if MediaWiki API _AND_ pywikipedia bot do not deprecate at once.
Now it looks as API: we are deprecating what we promised to deprecated long ago - ok pywikipedia compat: did not handle the deprecation of API before, and are not going to fix copy-pasted in tens of places (not one place, it's never that simple) query builders to support "rawcontinue", we announce compat as discontinued together with the old style API. API deprecation was not coordinated with client library deprecation - not ok If there is one year gap between two deprecations - ok, bot writers can choose either compat or core, and their bots can still work. Most users don't use APi directly so it should be the problem of coordination between API and clients developers. > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 3 Jun 2015 19:08:24 +0300 > From: Yuri Astrakhan <yastrak...@wikimedia.org> > To: Wikimedia developers <wikitech-l@lists.wikimedia.org> > Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation > mode for action=query will change at the end of this month > Message-ID: > <CALOOOkhzCUf14Tf= > p0uqw7zxamwb9qaxs4kgvbjavssmqy8...@mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > MZ: >I feel that former MediaWiki Web API maintainers should actively pay > attention to which mailing lists they're posting to. ;-) I doubt you > intended to send this message to mediawiki-api-announce. > Mz, I don't think I ever spent much time maintaining it :)) But yes, good > point, reply all is evil at times :) > > MZ: re why minimalistic lib - for most apis out there, people tend to write > "reference implementation" - code that explains to would-be lib authors how > to use API. It sets up prefered usage patterns and guidelines. We never had > that for the api. This resulted in a large number of API libs that vary in > how they use api. Pywikibot is a powerful platform, but is too big for many > usecases (as discussed in Lyon). Hence the need. > > SW: >Most operator are volunteers and don't have time to change the code > every month because there is a change in the api. > * I might agree about every month, but we are talking about a feature that > has been out for 2 years... > > > The old one was imho perfectly. > * Most API users imho would laugh at this statement. See the roadmap > <https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap> which > came as a result of analysing numerous bugs & wishes filed against the api, > such as returning {} instead of [] for empty values, inconsistencies, the > silly '*' value for text, and many other problems that have accumulated > during the years. API might work for you, but it does not support > multilingual error messaging, it overcomplicates the process of writing > javascript clients, it does not easily cache. In short, lots of problems. > > > Was the new api coded by WMF or by volunteers? > * I wrote the original API as a volunteer (took about a year in 2005/6). I > recently coded the continuation as a volunteer, and Brad has improved it as > a WMF employee. I later became a full time WMF employee as well, but my > project was Zero, not API. As a volunteer, over 2 years ago I wrote a huge > API > improvement roadmap > <https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap>that Brad > picked up and greatly extended, and is driving it forward with the amazing > improvements. From what I know, Brad has now been officially moved into > position of working on the API. In other words, that employee vs volunteer > line is very fuzzy. No point of splitting it that way. > > TLDR; > > In short, we need to make improvements if we want to move forward in > turning API into a platform, with flexible JavaScript-driven interfaces > such as Visual Editor. To allow creative uses that go beyond AWB, that > support complex interactions, and not just the way for bots to make edits. > Unfortunately, that means that despite our best efforts, very infrequently, > some bots must be updated. > > If the bot author cannot add a simple one line change "rawcontinue=1" once > in two years because of their busy personal live, I don't think that person > should be making automatic edits to Wikipedia - because sometimes bots make > mistakes that require substantially more time involvement. I would not > trust wikipedia edits to a bot runner under such circumstances. If the bot > runner is not a programmer, they should get the latest version of their > bot. If there is no up-to-date code because noone is maintaining it, it > again should not be accessing wikipedia - we sometimes discover security > bugs that require fixing, or because bot calls wiki too often, or other > changes in content structure - e.g. introduction of WikiData for interwiki > links required all interwiki bots to be updated. > > Wikipedia is a living, developing ecosystem, and it does require updates to > all parties accessing it. People accessing wikipedia from the older > browsers discover that they no longer can do everything they used to many > years ago - because we now use newer browser features, and fallback into > basic mode for older browsers. Please participate in the evolution of the > project. > > > On Wed, Jun 3, 2015 at 5:22 PM, Steinsplitter Wiki < > steinsplitter-w...@live.com> wrote: > > > Most operator are volunteers and don't have time to change the code every > > month because there is a change in the api. Because of this devs should > > keep the api backward-compatible. > > Also wondering why wee need this "new" api. The old one was imho > perfectly. > > > > Was the new api coded by WMF or by volunteers? > > > > > I feel that bot operators should actively pay attention to the > technical > > > aspects of the community and the mailing lists. > > Sorry, i disagree. Bot operators are volunteers and not payed staffers. > > Most of them having a job and real live. > > > > -- Steinsplitter > > > > > Date: Wed, 3 Jun 2015 14:50:48 +0300 > > > From: yastrak...@wikimedia.org > > > To: wikitech-l@lists.wikimedia.org > > > CC: mediawiki-api-annou...@lists.wikimedia.org > > > Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation > mode > > for action=query will change at the end of this month > > > > > > I feel that bot operators should actively pay attention to the > technical > > > aspects of the community and the mailing lists. So, the bot operator > who > > > never updates their software, doesn't pay attention to the > announcements, > > > and ignores api warnings should be blocked after the deadline. Bot > > > operators do not operate in a vacuum, and should never run bots just > for > > > the sake of running them. > > > Community should always be able to find and communicate with the bot > > > operators. > > > Obviously we should not make sudden changes (except in the > > > security/breaking matters), and try to make the process as easy as > > > possible. The rawcontinue param is exactly that, simply adding it will > > keep > > > the logic as before. > > > > > > Lastly, I again would like to promote the idea discussed at the > hackathon > > > -- a client side minimalistic library that bigger frameworks like > > pywikibot > > > rely on, and that is designed in part by the core developers. See the > > > proposal at > > > > > > https://www.mediawiki.org/wiki/Requests_for_comment/Minimalistic_MW_API_Client_Lib_Specification > > > On Jun 3, 2015 2:29 PM, "John Mark Vandenberg" <jay...@gmail.com> > wrote: > > > > > > > On Wed, Jun 3, 2015 at 3:42 AM, Brad Jorsch (Anomie) > > > > <bjor...@wikimedia.org> wrote: > > > > > ... > > > > > I've compiled a list of bots that have hit the deprecation warning > > more > > > > > than 10000 times over the course of the week May 23–29. If you are > > > > > responsible for any of these bots, please fix them. If you know who > > is, > > > > > please make sure they've seen this notification. Thanks. > > > > > > > > Thank you Brad for doing impact analysis and providing a list of the > > > > 71 bots with more than 10,000 problems per week. We can try to solve > > > > those by working with the bot operators. > > > > > > > > If possible, could you compile a list of bots affected at a lower > > > > threshold - maybe 1,000. That will give us a better idea of the > scale > > > > of bots operators that will be affected when this lands - currently > in > > > > one months time. > > > > > > > > Will the deploy date be moved back if the impact doesnt diminish by > > > > bots being fixed? > > > > > > > > -- > > > > John Vandenberg > > > > > > > > _______________________________________________ > > > > Wikitech-l mailing list > > > > Wikitech-l@lists.wikimedia.org > > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > _______________________________________________ > > > Wikitech-l mailing list > > > Wikitech-l@lists.wikimedia.org > > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > _______________________________________________ > > Wikitech-l mailing list > > Wikitech-l@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > > > ------------------------------ > > Message: 2 > Date: Wed, 3 Jun 2015 12:34:12 -0400 > From: "Brad Jorsch (Anomie)" <bjor...@wikimedia.org> > To: Wikimedia developers <wikitech-l@lists.wikimedia.org> > Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation > mode for action=query will change at the end of this month > Message-ID: > < > caeeprsukej9o0begt6xco+c+j7uta0btzoecaqs9cz6xq2g...@mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Wed, Jun 3, 2015 at 6:49 AM, Steinsplitter Wiki < > steinsplitter-w...@live.com> wrote: > > > I haven't followed this discussion, however i am wondering why api is not > > keep backward compatible. This will break a lot of stuff and i am > wondering > > why we need such changes in the [API] > > > > We usually do. In this case, however, the advantages of changing the > default for new API users seems to outweigh the disadvantages of a > well-announced breaking change with a simple parameter to request > backwards-compatible output. > > > -- > Brad Jorsch (Anomie) > Software Engineer > Wikimedia Foundation > > > ------------------------------ > > Message: 3 > Date: Wed, 3 Jun 2015 12:43:36 -0400 > From: "Brad Jorsch (Anomie)" <bjor...@wikimedia.org> > To: Wikimedia developers <wikitech-l@lists.wikimedia.org> > Cc: mediawiki-api-annou...@lists.wikimedia.org > Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation > mode for action=query will change at the end of this month > Message-ID: > < > caeeprsvprwamr3y+tffclg6jv4hkbtcqbzj6jxfsvze4v8f...@mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Wed, Jun 3, 2015 at 7:29 AM, John Mark Vandenberg <jay...@gmail.com> > wrote: > > > If possible, could you compile a list of bots affected at a lower > > threshold - maybe 1,000. That will give us a better idea of the scale > > of bots operators that will be affected when this lands - currently in > > one months time. > > > > I already have the list of *accounts* affected: there are 510 with between > 1000 and 10000 hits. Of those, 454 do not contain "bot" (case > insensitively), so they might be human users with user scripts, or AWB if > that's not fixed (someone please check!), or the like. For comparison, in > the over-10000 group there were 30 such that I filtered out. > > I'll want to check with Legal to make sure the additional release of > account names is still compliant with the privacy policy (I'm almost but > not entirely sure it would be ok). > > > > Will the deploy date be moved back if the impact doesnt diminish by > > bots being fixed? > > > > That's not impossible, but I wouldn't count on it. > > > -- > Brad Jorsch (Anomie) > Software Engineer > Wikimedia Foundation > > > ------------------------------ > > Message: 4 > Date: Wed, 3 Jun 2015 13:13:34 -0400 > From: "Brad Jorsch (Anomie)" <bjor...@wikimedia.org> > To: Wikimedia developers <wikitech-l@lists.wikimedia.org> > Cc: MediaWiki API announcements & discussion > <mediawiki-...@lists.wikimedia.org> > Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation > mode for action=query will change at the end of this month > Message-ID: > <CAEepRSsfin=praEeth2qQQFtwS_SHgFDbZFpXEd= > uqorvn4...@mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On Wed, Jun 3, 2015 at 10:04 AM, Brian Gerstle <bgers...@wikimedia.org> > wrote: > > > My question is: why does the default behavior need to change? Wouldn't > > continuing with the default behavior allow people to continue using the > > "rawcontinue" behavior for as long as we want to support it—without > making > > any changes? > > > > The decision to change the default came out of the same concerns that led > to the improved action=help output and some of the other work I've been > doing lately: We want to lower the barriers for using our API, which means > that the default shouldn't be something user-hostile. > > The raw continuation is deceptively simple: it looks straightforward, but > if you're using it with a generator, multiple prop modules, and meta or > list modules, your client code has to know when to ignore the returned > continuation for the generator, when to remove a module from prop and then > when to re-add it, and when to remove the meta or list modules. I wouldn't > be that surprised to learn that more people have it wrong than correct if > their code supports using prop modules with generators. > > The new continuation actually is simple: you send the equivalent of > array_merge( $originalParams, $continueParams ) and it just works. > > > Yes, some of the same could be said for making format=json&formatversion=2 > the default. In this case the formatversion=1 output is just annoying > rather than actually hostile (although representing boolean true as a > falsey string comes close), so at this time there's no plan to make that > breaking change. > > > -- > Brad Jorsch (Anomie) > Software Engineer > Wikimedia Foundation > > > ------------------------------ > > Message: 5 > Date: Wed, 3 Jun 2015 20:31:42 +0300 > From: Yuri Astrakhan <yastrak...@wikimedia.org> > To: "MediaWiki API announcements & discussion" > <mediawiki-...@lists.wikimedia.org> > Cc: Wikimedia developers <wikitech-l@lists.wikimedia.org> > Subject: Re: [Wikitech-l] [Mediawiki-api] API BREAKING CHANGE: Default > continuation mode for action=query will change at the end of this > month > Message-ID: > < > caloookjcnxoxcsmoqe5z58nr2iw24tf7qpcoksxwt1qzlt3...@mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > I would like to point out that it might be a good idea add > &formatversion=1 for anyone who wants to lock the current formatting in > place. > > On Wed, Jun 3, 2015 at 8:13 PM, Brad Jorsch (Anomie) < > bjor...@wikimedia.org> > wrote: > > > On Wed, Jun 3, 2015 at 10:04 AM, Brian Gerstle <bgers...@wikimedia.org> > > wrote: > > > >> My question is: why does the default behavior need to change? Wouldn't > >> continuing with the default behavior allow people to continue using the > >> "rawcontinue" behavior for as long as we want to support it—without > making > >> any changes? > >> > > > > The decision to change the default came out of the same concerns that led > > to the improved action=help output and some of the other work I've been > > doing lately: We want to lower the barriers for using our API, which > means > > that the default shouldn't be something user-hostile. > > > > The raw continuation is deceptively simple: it looks straightforward, but > > if you're using it with a generator, multiple prop modules, and meta or > > list modules, your client code has to know when to ignore the returned > > continuation for the generator, when to remove a module from prop and > then > > when to re-add it, and when to remove the meta or list modules. I > wouldn't > > be that surprised to learn that more people have it wrong than correct if > > their code supports using prop modules with generators. > > > > The new continuation actually is simple: you send the equivalent of > > array_merge( $originalParams, $continueParams ) and it just works. > > > > > > Yes, some of the same could be said for making > format=json&formatversion=2 > > the default. In this case the formatversion=1 output is just annoying > > rather than actually hostile (although representing boolean true as a > > falsey string comes close), so at this time there's no plan to make that > > breaking change. > > > > > > -- > > Brad Jorsch (Anomie) > > Software Engineer > > Wikimedia Foundation > > > > _______________________________________________ > > Mediawiki-api mailing list > > mediawiki-...@lists.wikimedia.org > > https://lists.wikimedia.org/mailman/listinfo/mediawiki-api > > > > > > > ------------------------------ > > Message: 6 > Date: Wed, 3 Jun 2015 11:55:40 -0600 > From: Brian Wolff <bawo...@gmail.com> > To: Wikimedia developers <wikitech-l@lists.wikimedia.org>, > mediawiki-...@lists.wikimedia.org > Subject: Re: [Wikitech-l] API BREAKING CHANGE: Default continuation > mode for action=query will change at the end of this month > Message-ID: > < > ca+oo+dunkpwl_esg7dmgqoq5osspjmh9fnl+ew_xbaoo6xf...@mail.gmail.com> > Content-Type: text/plain; charset=UTF-8 > > On 6/3/15, Steinsplitter Wiki <steinsplitter-w...@live.com> wrote: > > Most operator are volunteers and don't have time to change the code every > > month because there is a change in the api. Because of this devs should > keep > > the api backward-compatible. > > Also wondering why wee need this "new" api. The old one was imho > perfectly. > > > > Was the new api coded by WMF or by volunteers? > > > >> I feel that bot operators should actively pay attention to the technical > >> aspects of the community and the mailing lists. > > Sorry, i disagree. Bot operators are volunteers and not payed staffers. > Most > > of them having a job and real live. > > > > -- Steinsplitter > > > > My understanding is that most of the people who were using the > original continuation, were using it wrong, causing subtle bugs in > their script. Thus the existing implementation was wasting > considerable amount of volunteer bot developer time. In the long run > this change will hopefully reduce the total amount of time spent by > volunteer bot makers chasing weird bugs, at the expense of some short > term pain. > > Its always a challenge to balance backwards compatibility with fixing > things that are causing problems. I think the API team is keenly aware > of the frustrations that changes to the api cause, and try to make > sure that intentional breakage only happens when the benefits truly > outweigh the cons. > > -- > Bawolff > > > > ------------------------------ > > _______________________________________________ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > > > End of Wikitech-l Digest, Vol 143, Issue 8 > ****************************************** > _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
