Re: [Numpy-discussion] NEP 37: A dispatch protocol for NumPy-like modules

On 2/23/20 6:59 PM, Ralf Gommers wrote: > One of the main rationales for the whole NEP, and the argument in > multiple places > (https://numpy.org/neps/nep-0037-array-module.html#opt-in-vs-opt-out-for-users) > is that it's now opt-in while __array_function__ was opt-out. This isn't > really true -

Re: [Numpy-discussion] New DTypes: Are scalars a central concept in NumPy or not?

I have some thoughts on scalars from playing with ndarray ducktypes (__array_function__), eg a MaskedArray ndarray-ducktype, for which I wanted an associated "MaskedScalar" type. In summary, the ways scalars currently work makes ducktyping (duck-scalars) difficult: * numpy scalar types are not

Re: [Numpy-discussion] argmax() indexes to value

my thought was to try `take` or `take_along_axis`: ind = np.argmin(a, axis=1) np.take_along_axis(a, ind[:,None], axis=1) But those functions tend to simply fall back to fancy indexing, and are pretty slow. On my system plain fancy indexing is fastest: >>> %timeit a[np.arange(N),ind] 1.58

Re: [Numpy-discussion] round / set_printoptions discrepancy

On 9/13/19 9:26 AM, Eric Moore wrote: > See the notes section here.   > https://numpy.org/devdocs/reference/generated/numpy.around.html. > > This note was recently added in https://github.com/numpy/numpy/pull/14392 > > Eric Hmm, but this example with 16.055 shows the note still isn't quite

Re: [Numpy-discussion] new MaskedArray class

On 6/24/19 3:09 PM, Marten van Kerkwijk wrote: > Hi Allan, > > Thanks for bringing up the noclobber explicitly (and Stephan for asking > for clarification; I was similarly confused). > > It does clarify the difference in mental picture. In mine, the operation > would indeed be guaranteed to be

Re: [Numpy-discussion] new MaskedArray class

On 6/24/19 12:16 PM, Stephan Hoyer wrote: > On Mon, Jun 24, 2019 at 8:46 AM Allan Haldane <mailto:allanhald...@gmail.com>> wrote: > >  1. Making a "no-clobber" guarantee on the underlying data > > > Hi Allan -- could kindly clarify what you mean by

Re: [Numpy-discussion] new MaskedArray class

On 6/24/19 11:46 AM, Allan Haldane wrote: > A no-clobber guarantee makes your "iterative mask" example solvable in > an efficient (no-copy) way: > > mask, last_mask = False > while True: > dat_mean = np.mean(MaskedArray(data, mask)) > mask,

Re: [Numpy-discussion] new MaskedArray class

On 6/23/19 6:58 PM, Eric Wieser wrote: > I think we’d need to consider separately the operation on the mask > and on the data. In my proposal, the data would always do > |np.sum(array, where=~mask)|, while how the mask would propagate > might depend on the mask itself, > > I quite

Re: [Numpy-discussion] new MaskedArray class

safely achieved by simply creating a new MaskedArray with a different mask. If super-users want to access the ._data attribute they can, but I don't think it should be recommended. Normal users can use the ".filled" method, which by the way I implemented to optionally support retu

Re: [Numpy-discussion] new MaskedArray class

lots of copies, and forces copies to be explicit in user code. 2. disallowing direct modification of the mask lowers the "API surface area" making people's MaskedArray code less buggy and easier to read: Exposure of nonsense values by "unmasking" is one less possibility to

Re: [Numpy-discussion] new MaskedArray class

xt, when I have time. I'll either try to finish my ArrayCollection type, or try making a simple NDunit ducktype piggybacking on astropy's Unit. Best, Allan > > All the best, > > Marten > > > On Wed, Jun 19, 2019 at 5:45 PM Allan Haldane <mailto:allanhald...@gmail.com>

Re: [Numpy-discussion] new MaskedArray class

On 6/18/19 2:04 PM, Marten van Kerkwijk wrote: > > > On Tue, Jun 18, 2019 at 12:55 PM Allan Haldane <mailto:allanhald...@gmail.com>> wrote: > > > > This may be too much to ask from the initializer, but, if so, it still > > seems most useful if it is

Re: [Numpy-discussion] new MaskedArray class

ot released. To avoid duplication of effort, I've attached a diff of what I tried. I actually get a slight slowdown of about 10% by using where... If you make progress with the mixin, a push is welcome. I imagine a problem is going to be that np.isscalar doesn't work to detect duck scalars. Cheers, Allan

[Numpy-discussion] new MaskedArray class

Hi all, Chuck suggested we think about a MaskedArray replacement for 1.18. A few months ago I did some work on a MaskedArray replacement using `__array_function__`, which I got mostly working. It seems like a good time to bring it up for discussion now. See it at:

Re: [Numpy-discussion] Moving forward with value based casting

On 6/6/19 12:46 PM, Sebastian Berg wrote: > On Thu, 2019-06-06 at 11:57 -0400, Allan Haldane wrote: >> I think dtype-based casting makes a lot of sense, the problem is >> backward compatibility. >> >> Numpy casting is weird in a number of ways: The array + array casting

Re: [Numpy-discussion] Moving forward with value based casting

I think dtype-based casting makes a lot of sense, the problem is backward compatibility. Numpy casting is weird in a number of ways: The array + array casting is unexpected to many users (eg, uint64 + int64 -> float64), and the casting of array + scalar is different from that, and value based.

Re: [Numpy-discussion] Behaviour of copy for structured dtypes with gaps

I would be much more in favor of `copy` eliminating padding in the dtype, if dtypes with different paddings were considered equivalent. But they are not. Numpy has always treated dtypes with different padding bytes as not-equal, and prints them very differently: >>> a = np.array([1],

Re: [Numpy-discussion] proposal of new keywords for np.nan_to_num

Since there seem to be no objections, I think we're going ahead with this enhancement for np.nan_to_num. Cheers, Allan On 4/4/19 1:11 PM, kikocorreoso wrote: > Hi all, > > I propose to add some keywords to nan_to_num function. The addition do > not modify the actual behavior. Information

Re: [Numpy-discussion] the making of the NumPy roadmap

Thanks a lot for this summary and review, so us devs who weren't there are aware of off-list discussions. Thanks all for the brainstorming work too! Cheers, Allan On 12/7/18 12:23 AM, Ralf Gommers wrote: Hi all, A little while ago I wrote a blog post about the history of us constructing

Re: [Numpy-discussion] Attribute hiding APIs for PyArrayObject

On 10/30/18 5:04 AM, Matti Picus wrote: > TL;DR - should we revert the attribute-hiding constructs in > ndarraytypes.h and unify PyArrayObject_fields with PyArrayObject? > > > Background > > > NumPy 1.8 deprecated direct access to PyArrayObject fields. It made > PyArrayObject "opaque", and hid

Re: [Numpy-discussion] Reminder: weekly status meeting 31.10 at 12:00 pacific time

I'll try to make it, but can't guarantee. The last time there was discussion of the structured-array PRs which are currently held up, and I sort of promised to have a writeup of the issues. I put up a draft of that here: https://gist.github.com/ahaldane/6cd44886efb449f9c8d5ea012747323b

Re: [Numpy-discussion] BIDS/NumPy dev meetings, Wednesdays 12pm Pacific

I'll try to make it, especially as it looks like you want to discuss two of my PRs! :) I have a different meeting a bit before then which might run over though, so sorry ahead of time if I'm not there. Cheers, Allan On 10/16/18 5:26 PM, Stefan van der Walt wrote: > Hi everyone, > > This is a

Re: [Numpy-discussion] ndrange, like range but multidimensiontal

On 10/10/18 12:34 AM, Eric Wieser wrote: > One thing that worries me here - in python, |range(...)| in essence > generates a lazy |list| - so I’d expect |ndrange| to generate a lazy > |ndarray|. In practice, that means it would be a duck-type defining an > |__array__| method to evaluate it, and

Re: [Numpy-discussion] ndrange, like range but multidimensiontal

On 10/07/2018 10:32 AM, Mark Harfouche wrote: With `np.ndrange` it can look something like this: ``` c = np.empty((4, 4), dtype=object) # ... compute on c for i in np.ndrange(c.shape)[:, 1:-1]:     c[i] # = some operation on c[i] that depends on the index i ``` very pythonic, very familiar to

Re: [Numpy-discussion] PR Cleanup

On 09/25/2018 11:51 AM, Charles R Harris wrote: > Hi All, > > As usual, the top of the PR stack is getting all the attention. As a > start on cleaning up the old PRs, I'd like to suggest that all the > maintainers look at their (long) pending PRs and decide which they want > to keep, close those

Re: [Numpy-discussion] CI Testing

On 09/12/2018 08:14 PM, Tyler Reddy wrote: Would also be cool to have native ppc64 arch in CI, but I checked briefly with IBM and no such integration exists with github as far as the contact knew. Hi Tyler, ppc64 CI is available through OSUOSL. Earlier this year I requested a ppc64be test

Re: [Numpy-discussion] Adoption of a Code of Conduct

On 08/02/2018 09:25 AM, Ralf Gommers wrote: > > > On Thu, Aug 2, 2018 at 3:35 AM, Marten van Kerkwijk > mailto:m.h.vankerkw...@gmail.com>> wrote: > > I think a legalistic focus on the letter rather than the spirit of > the code of conduct is not that helpful (and probably what makes if

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On 05/31/2018 04:14 PM, Marten van Kerkwijk wrote: > I am -1 on multiple signatures. We may revisit this in time, but for > now I find the minimal intrusiveness of the current changes > appealing, especially as it requires few to no changes whatsoever to > the inner loop function.

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On 05/31/2018 12:10 AM, Nathaniel Smith wrote: On Wed, May 30, 2018 at 11:14 AM, Marten van Kerkwijk - When it comes to the core ufunc machinery, we have a limited complexity budget. I'm nervous that if we add too many bells and whistles, we'll end up writing ourselves into a corner where we

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On 05/31/2018 09:53 AM, Sebastian Berg wrote: > Also, I do not imagine these as free-floating ufuncs, I think we can arrange them in a logical way in a gufunc ecosystem. There would be some "core ufuncs", with "associated gufuncs" accessible as attributes. For instance, any_less_than will be

Re: [Numpy-discussion] Allowing broadcasting of code dimensions in generalized ufuncs

On 05/31/2018 12:10 AM, Nathaniel Smith wrote: On Wed, May 30, 2018 at 11:14 AM, Marten van Kerkwijk wrote: Hi All, Following on a PR combining the ability to provide fixed and flexible dimensions [1] (useful for, e.g., 3-vector input with a signature like `(3),(3)->(3)`, and for `matmul`,

Re: [Numpy-discussion] Splitting MaskedArray into a separate package

On 05/24/2018 11:31 AM, Sebastian Berg wrote: > I also somewhat like the idea of taking it out (once we have a first > replacement) in the case that we have a plan to do a better/lower level > replacement at a later point within numpy. > Removal generally has its merits, but if a (mid term)

Re: [Numpy-discussion] Splitting MaskedArray into a separate package

On 05/23/2018 04:02 PM, Eric Firing wrote: > Bad or missing values (and situations where one wants to use a mask to > operate on a subset of an array) are found in many domains of real life; > do you really want python users in those domains to have to fall back on > Matlab-style reliance on nans

Re: [Numpy-discussion] Extending ufunc signature syntax for matmul, frozen dimensions

On 04/29/2018 05:46 AM, Matti Picus wrote: > In looking to solve issue #9028 "no way to override matmul/@ if > __array_ufunc__ is set", it seems there is consensus around the idea of > making matmul a true gufunc, but matmul can behave differently for > different combinations of array and vector:

[Numpy-discussion] NumPy 1.14.3 released

cythonized with Cython 0.28.2. Contributors A total of 6 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Allan Haldane * Charles Harris * Jonathan March + * Malcolm Smith + * Matti Picus * Pauli Virtanen

Re: [Numpy-discussion] Short-circuiting equivalent of np.any or np.all?

On 04/26/2018 12:45 PM, Nathan Goldbaum wrote: > I'm curious if there's a way to get a fast early-terminating search in > NumPy? Perhaps there's another package I can depend on that does this? I > guess I could also write a bit of cython code that does this but so far > this project is pure python

Re: [Numpy-discussion] nditer as a context manager (reformatted?)

Given the lack of objections, we are probably going forward with this change to nditer. Anyone who uses nditers may have to update their code slightly if they want to avoid deprecation warnings, but otherwise old nditer code should work for a long time from now. Allan On 03/22/2018 01:43 PM,

Re: [Numpy-discussion] Using np.frombuffer and cffi.buffer on array of C structs (problem with struct member padding)

18 10:30 schrieb Joe: Thanks for your help on this! This solved my issue. Am 25.01.2018 um 19:01 schrieb Allan Haldane: There is a new section discussing alignment in the numpy 1.14 structured array docs, which has some hints about interfacing with C structs. These new 1.14 docs are not onl

Re: [Numpy-discussion] Setting custom dtypes and 1.14

On 01/30/2018 04:54 PM, josef.p...@gmail.com wrote: > > > On Tue, Jan 30, 2018 at 3:21 PM, Allan Haldane <allanhald...@gmail.com > <mailto:allanhald...@gmail.com>> wrote: > > On 01/30/2018 01:33 PM, josef.p...@gmail.com > <mailto:josef.p...@gmail.com

Re: [Numpy-discussion] Setting custom dtypes and 1.14

On 01/29/2018 11:50 PM, josef.p...@gmail.com wrote: On Mon, Jan 29, 2018 at 10:44 PM, Allan Haldane <allanhald...@gmail.com <mailto:allanhald...@gmail.com>> wrote: On 01/29/2018 05:59 PM, josef.p...@gmail.com <mailto:josef.p...@gmail.com> wrote: On Mon,

Re: [Numpy-discussion] Setting custom dtypes and 1.14

On 01/29/2018 05:59 PM, josef.p...@gmail.com wrote: On Mon, Jan 29, 2018 at 5:50 PM, <josef.p...@gmail.com <mailto:josef.p...@gmail.com>> wrote: On Mon, Jan 29, 2018 at 4:11 PM, Allan Haldane <allanhald...@gmail.com <mailto:allanhald...@gmail.com>> wrote:

Re: [Numpy-discussion] Setting custom dtypes and 1.14

On 01/29/2018 04:02 PM, josef.p...@gmail.com wrote: > > > On Mon, Jan 29, 2018 at 3:44 PM, Benjamin Root > wrote: > > I <3 structured arrays. I love the fact that I can access data by > row and then by fieldname, or vice versa. There

Re: [Numpy-discussion] Multiple-field indexing: view vs copy in 1.14+

On 01/25/2018 03:56 PM, josef.p...@gmail.com wrote: On Thu, Jan 25, 2018 at 1:49 PM, Marten van Kerkwijk > wrote: On Thu, Jan 25, 2018 at 1:16 PM, Stefan van der Walt >

Re: [Numpy-discussion] Setting custom dtypes and 1.14

On 01/25/2018 08:53 PM, Chris Barker - NOAA Federal wrote: >> On Jan 25, 2018, at 4:06 PM, Allan Haldane <allanhald...@gmail.com> wrote: > >>> 1) This is a known change with good reason? > >> . The >> change occurred because the old assignment behavior w

Re: [Numpy-discussion] Setting custom dtypes and 1.14

On 01/25/2018 06:06 PM, Chris Barker wrote: > Hi all, > > I'm pretty sure this is the same thing as recently discussed on this > list about 1.14, but to confirm: > > I had failures in my code with an upgrade for 1.14 -- turns out it was a > single line in a single test fixture, so no big deal,

Re: [Numpy-discussion] Using np.frombuffer and cffi.buffer on array of C structs (problem with struct member padding)

There is a new section discussing alignment in the numpy 1.14 structured array docs, which has some hints about interfacing with C structs. These new 1.14 docs are not online yet on scipy.org, but in the meantime you can view them here:

Re: [Numpy-discussion] Multiple-field indexing: view vs copy in 1.14+

On 01/22/2018 11:23 AM, Allan Haldane wrote: > I just want to note that I > thought of another way to "fix" this for 1.15 which does not involve > "pack_fields", which is > >     a[['b', 'c']].astype('f8,f8').view(('f8', 2)) > > Which is back-c

Re: [Numpy-discussion] Multiple-field indexing: view vs copy in 1.14+

On 01/22/2018 10:53 AM, josef.p...@gmail.com wrote: On Sun, Jan 21, 2018 at 9:48 PM, Allan Haldane <allanhald...@gmail.com In  many examples where I used structured dtypes a long time ago, switched between consistent views as either a standard array of subsets or as .structured dty

Re: [Numpy-discussion] Multiple-field indexing: view vs copy in 1.14+

On 01/22/2018 10:53 AM, josef.p...@gmail.com wrote: This is similar to the above example a[['a', 'c']].view('i8') but it doesn't try to combine fields. In  many examples where I used structured dtypes a long time ago, switched between consistent views as either a standard array of subsets or

[Numpy-discussion] Multiple-field indexing: view vs copy in 1.14+

Hello all, We are making a decision (again) about what to do about the behavior of multiple-field indexing of structured arrays: Should it return a view or a copy, and on what release schedule? As a reminder, this refers to operations like (1.13 behavior): >>> a = np.zeros(3, dtype=[('a',

[Numpy-discussion] PowerPC machine available from OSUOSL

Hi all, For building and testing Numpy on the PowerPC arch, I've requested and obtained a VM instance at OSUOSL, which provides free hosting for open-source projects. This was suggested by @npanpaliya on github. http://osuosl.org/services/powerdev/request_hosting/ I have an immediate use for it

Re: [Numpy-discussion] PyArray_GETITEM and PyArray_SETITEM

On 11/13/2017 01:53 PM, Mmanu Chaturvedi wrote: > Hello All, > > I need to make use of the limited numpy API access Pybind11 gives, in order > to add a feature to it. It seems to give access to functions from > numpy_api.py [1]. I need to use PyArray_GETITEM and PyArray_SETITEM in > order to

Re: [Numpy-discussion] np.vstack vs. np.stack

On 11/09/2017 05:39 PM, Robert Kern wrote: > On Thu, Nov 9, 2017 at 1:58 PM, Mark Bakker wrote: > >> Can anybody explain why vstack is going the way of the dodo? >> Why are stack / concatenate better? What is 'bad' about vstack? > > As far as I can tell, the discussion

Re: [Numpy-discussion] np.vstack vs. np.stack

On 11/09/2017 04:30 AM, Joe wrote: > Hello, > > I have a question and hope that you can help me. > > The doc for vstack mentions that "this function continues to be > supported for backward compatibility, but you should prefer > np.concatenate or np.stack." > > Using vstack was convenient

Re: [Numpy-discussion] deprecate updateifcopy in nditer operand, flags?

On 11/08/2017 03:12 PM, Nathaniel Smith wrote: > - We could adjust the API so that there's some explicit operation to > trigger the final writeback. At the Python level this would probably > mean that we start supporting the use of nditer as a context manager, > and eventually start raising an

Re: [Numpy-discussion] np.array, copy=False and memmap

On 08/10/2017 02:24 PM, Sebastian Berg wrote: > On Thu, 2017-08-10 at 12:27 -0400, Allan Haldane wrote: >> On 08/07/2017 05:01 PM, Nisoli Isaia wrote: >>> Dear all, >>> I have a question about the behaviour of >>> >>> y = np.array(x, copy=False, dtype

Re: [Numpy-discussion] Scipy 2017 NumPy sprint

On 07/02/2017 10:03 AM, Charles R Harris wrote: Updated list below. On Sat, Jul 1, 2017 at 7:08 PM, Benjamin Root > wrote: Just a heads-up. There is now a sphinx-gallery plugin. Matplotlib and a few other projects have migrated their

Re: [Numpy-discussion] proposed changes to array printing in 1.14

m the earlier change in printing for structured arrays >> (which was also much for the better, but broke a lot of doctests). > >> -- Marten > > > >> On Thu, Jun 29, 2017 at 3:38 PM, Allan Haldane <allanhald...@gmail.com> >> wrote: > >>