On Thu, Aug 13, 2020 at 5:22 AM Ralf Gommers <ralf.gomm...@gmail.com> wrote:
> Thanks for raising these concerns Ilhan and Juan, and for answering Peter. > Let me give my perspective as well. > > To start with, this is not specifically about Peter's NEP and PR. NEP 35 > simply follows the pattern set by previous PRs, and given its tight scope > is less difficult to understand than other NEPs on such technical topics. > Peter has done a lot of things right, and is close to the finish line. > > > On Thu, Aug 13, 2020 at 12:02 PM Peter Andreas Entschev < > pe...@entschev.com> wrote: > >> >> > I think, arriving to an agreement would be much faster if there is an >> executive summary of who this is intended for and what the regular usage >> is. Because with no offense, all I see is "dispatch", "_array_function_" >> and a lot of technical details of which I am absolutely ignorant. >> >> This is what I intended to do in the Usage Guidance [2] section. Could >> you elaborate on what more information you'd want to see there? Or is >> it just a matter of reorganizing the NEP a bit to try and summarize >> such things right at the top? >> > > We adapted the NEP template [6] several times last year to try and improve > this. And specified in there as well that NEP content set to the mailing > list should only contain the sections: Abstract, Motivation and Scope, > Usage and Impact, and Backwards compatibility. This to ensure we fully > understand the "why" and "what" before the "how". Unfortunately that > template and procedure hasn't been exercised much yet, only in NEP 38 [7] > and partially in NEP 41 [8]. > > If we have long-time maintainers of SciPy (Ilhan and myself), scikit-image > (Juan) and CuPy (Leo, on the PR review) all saying they don't understand > the goals, relevance, target audience, or how they're supposed to use a new > feature, that indicates that the people doing the writing and having the > discussion are doing something wrong at a very fundamental level. > > At this point I'm pretty disappointed in and tired of how we write and > discuss NEPs on technical topics like dispatching, dtypes and the like. > People literally refuse to write down concrete motivations, goals and > non-goals, code that's problematic now and will be better/working post-NEP > and usage examples before launching into extensive discussion of the gory > details of the internals. I'm not sure what to do about it. Completely > separate API and behavior proposals from implementation proposals? Make > separate "API" and "internals" teams with the likes of Juan, Ilhan and Leo > on the API team which then needs to approve every API change in new NEPs? > Offer to co-write NEPs if someone is willing but doesn't understand how to > go about it? Keep the current structure/process but veto further approvals > until NEP authors get it right? > I think the NEP template is great, and we should try to be more diligent about following it! My own NEP 37 (__array_module__) is probably a good example of poor presentation due to not following the template structure. It goes pretty deep into low-level motivation and some implementation details before usage examples. Speaking just for myself, I would have appreciated a friendly nudge to use the template. Certainly I think it would be fine to require using the template for newly submitted NEPs. I did not remember about it when I started drafting NEP 37, and it definitely would have helped. I may still try to do a revision at some point to use the template structure. > I want to make an exception for merging the current NEP, for which the > plan is to merge it as experimental to try in downstream PRs and get more > experience. That does mean that master will be in an unreleasable state by > the way, which is unusual and it'd be nice to get Chuck's explicit OK for > that. But after that, I think we need a change here. I would like to hear > what everyone thinks is the shape that change should take - any of my above > suggestions, or something else? > > > >> > Finally as a minor point, I know we are mostly (ex-)academics but this >> necessity of formal language on NEPs is self-imposed (probably PEPs are to >> blame) and not quite helping. It can be a bit more descriptive in my >> external opinion. >> >> TBH, I don't really know how to solve that point, so if you have any >> specific suggestions, that's certainly welcome. I understand the >> frustration for a reader trying to understand all the details, with >> many being only described in NEP-18 [3], but we also strive to avoid >> rewriting things that are written elsewhere, which would also >> overburden those who are aware of what's being discussed. >> >> >> > I also share Ilhan’s concern (and I mentioned this in a previous NEP >> discussion) that NEPs are getting pretty inaccessible. In a sense these are >> difficult topics and readers should be expected to have *some* familiarity >> with the topics being discussed, but perhaps more effort should be put into >> the context/motivation/background of a NEP before accepting it. One way to >> ensure this might be to require a final proofreading step by someone who >> has not been involved at all in the discussions, like peer review does for >> papers. >> > > Some variant of this proposal would be my preference. > > Cheers, > Ralf > > >> [1] https://github.com/numpy/numpy/issues/14441#issuecomment-529969572 >> [2] >> https://numpy.org/neps/nep-0035-array-creation-dispatch-with-array-function.html#usage-guidance >> [3] https://numpy.org/neps/nep-0018-array-function-protocol.html >> [4] https://numpy.org/neps/nep-0000.html#nep-workflow >> [5] >> https://mail.python.org/pipermail/numpy-discussion/2019-October/080176.html > > > [6] https://github.com/numpy/numpy/blob/master/doc/neps/nep-template.rst > [7] > https://github.com/numpy/numpy/blob/master/doc/neps/nep-0038-SIMD-optimizations.rst > [8] > https://github.com/numpy/numpy/blob/master/doc/neps/nep-0041-improved-dtype-support.rst > > > >> >> >> On Thu, Aug 13, 2020 at 3:44 AM Juan Nunez-Iglesias <j...@fastmail.com> >> wrote: >> > >> > I’ve generally been on the “let the NumPy devs worry about it” side of >> things, but I do agree with Ilhan that `like=` is confusing and `typeof=` >> would be a much more appropriate name for that parameter. >> > >> > I do think library writers are NumPy users and so I wouldn’t really >> make that distinction, though. Users writing their own analysis code could >> very well be interested in writing code using numpy functions that will >> transparently work when the input is a CuPy array or whatever. >> > >> > I also share Ilhan’s concern (and I mentioned this in a previous NEP >> discussion) that NEPs are getting pretty inaccessible. In a sense these are >> difficult topics and readers should be expected to have *some* familiarity >> with the topics being discussed, but perhaps more effort should be put into >> the context/motivation/background of a NEP before accepting it. One way to >> ensure this might be to require a final proofreading step by someone who >> has not been involved at all in the discussions, like peer review does for >> papers. >> > >> > Food for thought. >> > >> > Juan. >> > >> > On 13 Aug 2020, at 9:24 am, Ilhan Polat <ilhanpo...@gmail.com> wrote: >> > >> > For what is worth, as a potential consumer in SciPy, it really doesn't >> say anything (both in NEP and the PR) about how the regular users of NumPy >> will benefit from this. If only and only 3rd parties are going to benefit >> from it, I am not sure adding a new keyword to an already confusing >> function is the right thing to do. >> > >> > Let me clarify, >> > >> > - This is already a very (I mean extremely very) easy keyword name to >> confuse with ones_like, zeros_like and by its nature any other >> interpretation. It is not signalling anything about the functionality that >> is being discussed. I would seriously consider reserving such obvious names >> for really obvious tasks. Because you would also expect the shape and ndim >> would be mimicked by the "like"d argument but it turns out it is acting >> more like "typeof=" and not "like=" at all. Because if we follow the >> semantics it reads as "make your argument asarray like the other thing" but >> it is actually doing, "make your argument an array with the other thing's >> type" which might not be an array after all. >> > >> > - Again, if this is meant for downstream libraries (because that's what >> I got out of the PR discussion, cupy, dask, and JAX were the only examples >> I could read) then hiding it in another function and writing with capital >> letters "this is not meant for numpy users" would be a much more convenient >> way to separate the target audience and regular users. >> numpy.astypedarray([[some data], [...]], type_of=x) or whatever else it may >> be would be quite clean and to the point with no ambiguous keywords. >> > >> > I think, arriving to an agreement would be much faster if there is an >> executive summary of who this is intended for and what the regular usage >> is. Because with no offense, all I see is "dispatch", "_array_function_" >> and a lot of technical details of which I am absolutely ignorant. >> > >> > Finally as a minor point, I know we are mostly (ex-)academics but this >> necessity of formal language on NEPs is self-imposed (probably PEPs are to >> blame) and not quite helping. It can be a bit more descriptive in my >> external opinion. >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
