Re: [Numpy-discussion] Moving forward with value based casting

2019-06-21 Thread Sebastian Berg
On Wed, 2019-06-05 at 21:35 -0400, Marten van Kerkwijk wrote: > Hi Sebastian, > > Tricky! It seems a balance between unexpected memory blow-up and > unexpected wrapping (the latter mostly for integers). > > Some comments specifically on your message first, then some more > general related ones.

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-18 Thread Sebastian Berg
Hi Hameer, On Tue, 2019-06-18 at 04:28 +0200, Hameer Abbasi wrote: > On Wed, 2019-06-12 at 12:55 -0500, Sebastian Berg wrote: > > On Wed, 2019-06-05 at 15:41 -0500, Sebastian Berg wrote: > > > Hi all, > > > > A type is safely castable to another if all of these numbers are > exceeded or met. >

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-17 Thread Hameer Abbasi
On Tue, 2019-06-18 at 04:28 +0200, Hameer Abbasi wrote: > On Wed, 2019-06-12 at 12:55 -0500, Sebastian Berg wrote: > > On Wed, 2019-06-05 at 15:41 -0500, Sebastian Berg wrote: > > > Hi all, > > > > > > TL;DR: > > > > > > Value based promotion seems complex both for users and ufunc- > > >

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-17 Thread Hameer Abbasi
On Wed, 2019-06-12 at 12:55 -0500, Sebastian Berg wrote: > On Wed, 2019-06-05 at 15:41 -0500, Sebastian Berg wrote: > > Hi all, > > > > TL;DR: > > > > Value based promotion seems complex both for users and ufunc- > > dispatching/promotion logic. Is there any way we can move forward > > here, > >

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-07 Thread Sebastian Berg
On Fri, 2019-06-07 at 13:19 -0500, Sebastian Berg wrote: > On Fri, 2019-06-07 at 07:18 +0200, Ralf Gommers wrote: > > > > On Fri, Jun 7, 2019 at 1:37 AM Nathaniel Smith > > wrote: > > > My intuition is that what users actually want is for *native > > > Python > > > types* to be treated as having

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-07 Thread Sebastian Berg
On Thu, 2019-06-06 at 19:34 -0400, Allan Haldane wrote: > On 6/6/19 12:46 PM, Sebastian Berg wrote: > > On Thu, 2019-06-06 at 11:57 -0400, Allan Haldane wrote: > > > I think dtype-based casting makes a lot of sense, the problem is > > > backward compatibility. > > > > > > Numpy casting is weird

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-07 Thread Sebastian Berg
On Fri, 2019-06-07 at 07:18 +0200, Ralf Gommers wrote: > > > On Fri, Jun 7, 2019 at 1:37 AM Nathaniel Smith wrote: > > My intuition is that what users actually want is for *native Python > > types* to be treated as having 'underspecified' dtypes, e.g. int is > > happy to coerce to

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-07 Thread Neal Becker
On Wed, Jun 5, 2019 at 4:42 PM Sebastian Berg wrote: I think the best approach is that if the user gave unambiguous types as inputs to operators then the output should be the same dtype, or type corresponding to the common promotion type of the inputs. If the input type is not specified, I

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-07 Thread Marten van Kerkwijk
On Fri, Jun 7, 2019 at 1:19 AM Ralf Gommers wrote: > > > On Fri, Jun 7, 2019 at 1:37 AM Nathaniel Smith wrote: > >> >> My intuition is that what users actually want is for *native Python >> types* to be treated as having 'underspecified' dtypes, e.g. int is >> happy to coerce to

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Ralf Gommers
On Fri, Jun 7, 2019 at 1:37 AM Nathaniel Smith wrote: > > My intuition is that what users actually want is for *native Python > types* to be treated as having 'underspecified' dtypes, e.g. int is > happy to coerce to int8/int32/int64/whatever, float is happy to coerce > to

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Nathaniel Smith
I haven't read all the thread super carefully, so I might have missed something, but I think we might want to look at this together with the special rule for scalar casting. IIUC, the basic end-user problem that motivates all thi sis: when you have a simple Python constant whose exact dtype is

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Allan Haldane
On 6/6/19 12:46 PM, Sebastian Berg wrote: > On Thu, 2019-06-06 at 11:57 -0400, Allan Haldane wrote: >> I think dtype-based casting makes a lot of sense, the problem is >> backward compatibility. >> >> Numpy casting is weird in a number of ways: The array + array casting >> is >> unexpected to many

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Sebastian Berg
On Thu, 2019-06-06 at 11:57 -0400, Allan Haldane wrote: > I think dtype-based casting makes a lot of sense, the problem is > backward compatibility. > > Numpy casting is weird in a number of ways: The array + array casting > is > unexpected to many users (eg, uint64 + int64 -> float64), and the >

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Allan Haldane
I think dtype-based casting makes a lot of sense, the problem is backward compatibility. Numpy casting is weird in a number of ways: The array + array casting is unexpected to many users (eg, uint64 + int64 -> float64), and the casting of array + scalar is different from that, and value based.

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Sebastian Berg
On Wed, 2019-06-05 at 21:35 -0400, Marten van Kerkwijk wrote: > Hi Sebastian, > > Tricky! It seems a balance between unexpected memory blow-up and > unexpected wrapping (the latter mostly for integers). > > Some comments specifically on your message first, then some more > general related ones.

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Sebastian Berg
On Wed, 2019-06-05 at 17:14 -0700, Tyler Reddy wrote: > A few thoughts: > > - We're not trying to achieve systematic guards against integer > overflow / wrapping in ufunc inner loops, right? The performance > tradeoffs for a "result-based" casting / exception handling addition > would presumably

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-06 Thread Ralf Gommers
On Wed, Jun 5, 2019 at 10:42 PM Sebastian Berg wrote: > Hi all, > > TL;DR: > > Value based promotion seems complex both for users and ufunc- > dispatching/promotion logic. Is there any way we can move forward here, > and if we do, could we just risk some possible (maybe not-existing) > corner

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-05 Thread Marten van Kerkwijk
Hi Sebastian, Tricky! It seems a balance between unexpected memory blow-up and unexpected wrapping (the latter mostly for integers). Some comments specifically on your message first, then some more general related ones. 1. I'm very much against letting `a + b` do anything else than `np.add(a,

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-05 Thread Tyler Reddy
A few thoughts: - We're not trying to achieve systematic guards against integer overflow / wrapping in ufunc inner loops, right? The performance tradeoffs for a "result-based" casting / exception handling addition would presumably be controversial? I know there was some discussion about having an

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-05 Thread Sebastian Berg
Hi all, Maybe to clarify this at least a little, here are some examples for what currently happen and what I could imagine we can go to (all in terms of output dtype). float32_arr = np.ones(10, dtype=np.float32) int8_arr = np.ones(10, dtype=np.int8) uint8_arr = np.ones(10, dtype=np.uint8)

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-05 Thread Sebastian Berg
On Wed, 2019-06-05 at 14:14 -0700, Stephan Hoyer wrote: > On Wed, Jun 5, 2019 at 1:43 PM Sebastian Berg < > sebast...@sipsolutions.net> wrote: > > Hi all, > > > > > > Because `uint8(127)` can also be a `int8`, but uint8(128) it is not > > as > > simple as finding the "minimal" dtype once and

Re: [Numpy-discussion] Moving forward with value based casting

2019-06-05 Thread Stephan Hoyer
On Wed, Jun 5, 2019 at 1:43 PM Sebastian Berg wrote: > Hi all, > > TL;DR: > > Value based promotion seems complex both for users and ufunc- > dispatching/promotion logic. Is there any way we can move forward here, > and if we do, could we just risk some possible (maybe not-existing) > corner

[Numpy-discussion] Moving forward with value based casting

2019-06-05 Thread Sebastian Berg
Hi all, TL;DR: Value based promotion seems complex both for users and ufunc- dispatching/promotion logic. Is there any way we can move forward here, and if we do, could we just risk some possible (maybe not-existing) corner cases to break early to get on the way? --- Currently when you