On Fri, Apr 10, 2020 at 7:18 PM Sebastian Berg <sebast...@sipsolutions.net> wrote:
> On Fri, 2020-04-10 at 18:19 +0200, Ralf Gommers wrote: > > On Fri, Apr 10, 2020 at 3:03 PM Sebastian Berg < > > sebast...@sipsolutions.net> > > wrote: > > > > > On Fri, 2020-04-10 at 12:27 +0200, Ralf Gommers wrote: > > > > > 3. I added features to allow transitioning:: > > > > > > > > > > get_array_module(*arrays, modules="numpy", > > > > > future_modules=("dask.array", "cupy"), > > > > > fallback="warn") > > > > > > > > > > Will give FutureWarning/DeprecationWarning where necessary, > > > > > in > > > > the > > > > > above "numpy" is supported, dask and cupy are supported but > > > > > not > > > > > enabled by default. `None` works to say "all modules". > > > > > Once the transition is done, just move dask and cupy into > > > > `modules` > > > > > and remove `fallback=None`. > > > > > > > > > > > > > So future_modules explicitly excludes compatible libraries that > > > > are > > > > not > > > > listed. Why would you want anyone to do that? I don't understand > > > > "supported > > > > but not enabled", and it looks undesirable to me to special-case > > > > any > > > > library in this mechanism. > > > > > > We hav two (or three) types of modules (either could be "all"). > > > > > > > I think we only have modules that implement __array_module__, and > > ones that > > don't. > > > > > > > 1. Supported modules that we dispatch to. > > > 2. Modules that are supported but will be dispatched to by default > > > only > > > in the future. So if the user got a future_module, they will get > > > a > > > FutureWarning. They have to chose to cast the inputs or opt-in > > > to > > > the future behaviour. > > > 3. Unsupported modules: If this is resolved it is an error. I > > > currently > > > assume that this does not need to be a negative list. > > > > > > You need to distinguish those somehow, since you need a way to > > > transition. Even if you expect that modules would always be *all* > > > modules, `numpy` is still the only accepted module originally. > > > > > > So, as I said, `future_modules` is only about transitioning and > > > enabling `FutureWarning`s. Does not have to live there, but we need > > > a > > > way to transition. > > > > > > > Sorry, I still don't get it - transition what? You seem to be > > operating on > > the assumption that the users of get_array_module want or need to > > control > > which numpy-like libraries they allow and which they don't. That > > seems > > fundamentally wrong. How would you treat, for example, an array > > library > > that is developed privately inside some company? > > > > Well, you still need to transition from NumPy -> allow everything, so > for now please just ignore that part if you like and use/assume: > > get_array_module(..., > modules="numpy", future_modules=None, fallback="warn") > > during the transition, and: > > get_array_module(...) > > after it. After all this is a draft-project right now, so it is just as > much about trying out what can be done. > It is not unlikely that this transition burden will be put more on the > library in any case, but it shows that it can be done. > > > As to my "fundamentally wrong" assumption. Should libraries goal be to > support everything? Definitely! > > But... I do not want to make that decision for libraries, so I if > library authors tell me that they have no interest in it, all the > better. Until then I am more than happy to keep that option on the > table. If just as a thought for library authors to consider their > options. > > Possible, brainstorming, reasons could be: > > 1. Say I currently heavily use cython code, so I am limited to NumPy > (or at least arrays that can expose a buffer/`__array_interface__`). > Now if someone adds a CUDA implementation, I would support cupy arrays, > but not distributed arrays. > I admit maybe checking that at function entry like this is the wrong > approach there. > If you need a particular feature, then checking for that feature (e.g. `hasattr(__array_interface__)`, and same for __cuda_array_interface__) seems like the right thing to do. Ralf > 2. To limit to certain types is to say "We know (and test) that our > library works with xarray, Dask, NumPy, and CuPy". Now you can say that > is also a misconception, because if you stick to just NumPy API you > should know that it will "just work" with everything. But in practice > it seems like it might happen? > In that case you may want to actually allow any odd array and just put > a warning, a bit like the transition warnings I put in for testing. > > > --- > > There are two other things I am wondering about. > > 1. Subclasses may want to return their superclasses module (even by > default?), in which case their behaviour depends on the superclass > module behaviour. Further a library would need to use `np.asanyarray()` > to prevent the subclass from being cast to the superclass. > > 2. There is one transition that does not quite exists. What if an > array-like starts implementing or expands `array-module`? > That seems fine, but in that case the array-like will have to provide > the `opt-in` context manager with a FutureWarning. > The transition from no `__array_module__` to implementing it may need > some thought, but I expect it is fine: The array-like simply always > gives a FutureWarning, although it cannot know what will actually > happen in the future (no change, error, or array-like takes control). > > - Sebastian > > > > Cheers, > > Ralf > > > > > > > > > These options do not have to be handled by us, it only helps here > > > with > > > having context managers to opt-in to new behaviour, and maybe to > > > get an > > > idea for how transitions can look like. > > > Alternatively, we could all to create project specific context > > > managers > > > to do the same and avoid possible scoping issues even more. > > > > > > - Sebastian > > > _______________________________________________ > > > NumPy-Discussion mailing list > > > NumPy-Discussion@python.org > > > https://mail.python.org/mailman/listinfo/numpy-discussion > > > > > > > _______________________________________________ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion