On Wed, Mar 16, 2016 at 7:32 PM, Fernando Perez <fperez....@gmail.com> wrote: > On Wed, Mar 16, 2016 at 3:45 PM, Steve Waterbury <water...@pangalactic.us> > wrote: >> >> On 03/16/2016 06:28 PM, Nathaniel Smith wrote: >>> >>> ... Sounds like a real deprecation cycle would have been better. >> >> >> IMHO for a library as venerable and widely-used as Numpy, a >> deprecation cycle is almost always better ... consider this a >> lesson learned. > > > Mandatory XKCD - https://xkcd.com/1172 > > We recently had a discussion about a similar "nobody we know uses nor should > use this api" situation in IPython, and ultimately decided that xkcd 1172 > would hit us, so opted in this case just for creating new cleaner APIs + > possibly doing slow deprecation of the old stuff. > > For a widely used library, if the code exists then someone, somewhere > depends on it, regardless of how broken or obscure you think the feature is. > We just have to live with that.
Sure, the point is well taken, and we've been working hard to make numpy's change policy more consistent, rigorous, and useful -- and this remains very much a work in progress. But engineering is fundamentally the art of making trade-offs, which are always messy and context-dependent. I actually rather like XKCD 1172 because it's a Rorschach test -- you can just as easily read it as saying that you should start by accepting that all changes will break something, and then move on to the more interesting discussion of which things are you going to break and what are the trade-offs. :-) Like, in this case: Our general policy is definitely to use a deprecation cycle. And if you look at the discussions back in September, we also considered options like deprecating-then-removing the broadcasting, or adding a new gufunc-registration-API that would enable the new behavior while preserving the old behavior for old users. Both sound appealing initially. But they both also have serious downsides: they mean preserving broken behavior, possibly indefinitely, which *also* breaks users' code. Notice that if we add a new API in 1.10, then most people won't actually switch until 1.10 becomes the *minimum* version they support, except they probably will forget then too. And we have to maintain both APIs indefinitely. And I'm dubious that the kind of anonymous corporate users who got broken by this would have noticed a deprecation warning -- during the 1.11 cycle we had to go around pointing out some year+ old deprecation warnings to all our really active and engaged downstreams, never mind the ones we only hear about months later through Travis :-/. In this particular case, as far as we could tell, all single existing users were actually *broken* by the current behavior, so it was a question of whether we should fix a bunch of real cases but risk breaking some rare/theoretical ones, or should we leave a bunch of real cases broken for a while in order to be gentler to the rare/theoretical ones. And would we have ever even learned about these cases that Travis's clients are running into if we hadn't broken things? And so forth. It's obviously true that we make mistakes and should try to learn from them to do better in the future, but I object to the idea that there are simple and neat answers available :-). (And sometimes in my more pessimistic moments I feel like a lot of the conventional rules for change management are really technology for shifting around blame :-/. "We had a deprecation period that you didn't notice; your code broke the same either way, but our use of a deprecation period means that now it's *your* fault". Or, in the opposite situation: "Sure, this API doesn't work in the way that anyone would actually expect or want, but if we fix it then it will be *our* fault when existing code breaks -- OTOH if we leave the brokenness there but document it, then it'll be *your* fault when you -- totally predictably -- fall into the trap we've left for you". IMO in both situations it's much healthier to skip the blame games and take responsibility for the actual end result whatever it is, and if that means that some kind of failure is inevitable, then oh well, lets think about how to optimize our process for minimum net failure given imperfect information and finite resources :-). Is there some way we can help downstreams notice deprecations earlier? It's a lot easier to measure the cost of making a change than of not making a change -- is there some way we can gather more data to correct for this bias? IMO *these* are the really interesting questions, and they're ones that we've been actively working on.) -n -- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion