Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-02 Thread Nathaniel Smith
On Fri, Jul 1, 2011 at 10:07 PM, Eric Firing efir...@hawaii.edu wrote: On 07/01/2011 06:40 PM, Nathaniel Smith wrote: On Fri, Jul 1, 2011 at 9:29 AM, Christopher Jordan-Squire BTW, you can't access the memory of a masked value by taking a view, at least if I'm reading this version of the NEP

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-02 Thread Benjamin Root
On Fri, Jul 1, 2011 at 11:40 PM, Nathaniel Smith n...@pobox.com wrote: I'm not sure what you mean here. If we have masked array support at all (and some people seem to want it), then we have to say more than it's an array with a mask. Indexing such a beast has to do *something*, so we need

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-02 Thread josef . pktd
On Sat, Jul 2, 2011 at 4:10 PM, Benjamin Root ben.r...@ou.edu wrote: On Fri, Jul 1, 2011 at 11:40 PM, Nathaniel Smith n...@pobox.com wrote: I'm not sure what you mean here. If we have masked array support at all (and some people seem to want it), then we have to say more than it's an array

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 2:36 AM, Keith Goodman kwgood...@gmail.com wrote: On Thu, Jun 30, 2011 at 10:51 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett matthew.br...@gmail.com wrote: In the interest of making the discussion as concrete as possible,

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 3:09 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jul 1, 2011 at 2:36 AM, Keith Goodman kwgood...@gmail.com wrote: On Thu, Jun 30, 2011 at 10:51 AM, Nathaniel Smith

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Nathaniel Smith
On Fri, Jul 1, 2011 at 7:09 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com wrote: Do you see problems with the alterNEP proposal? Yes, I really like my design as it stands now, and the alterNEP removes a lot of the abstraction

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Christopher Barker
Matthew Brett wrote: should raise an error. On the other hand, if I make a normal array: arr = np.array([1.0, 2.0, 7.0]) and then do this: arr.visible[2] = False then either I should raise an error (it's not a masked array), or, more magically, construct a mask on the fly. maybe

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Mark Wiebe
On Fri, Jul 1, 2011 at 9:50 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Fri, Jul 1, 2011 at 3:09 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jul 1, 2011 at 2:36 AM, Keith Goodman

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Charles R Harris
On Fri, Jul 1, 2011 at 9:34 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:50 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Fri, Jul 1, 2011 at 3:09 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 4:34 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:50 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jul 1, 2011 at 3:09 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 4:48 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:34 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:50 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jul 1, 2011 at 3:09 PM, Mark Wiebe

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Charles R Harris
On Fri, Jul 1, 2011 at 10:00 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Fri, Jul 1, 2011 at 4:34 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:50 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jul 1, 2011 at 3:09 PM, Mark Wiebe

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Benjamin Root
On Fri, Jul 1, 2011 at 11:00 AM, Matthew Brett matthew.br...@gmail.comwrote: You can't switch between the two approaches without big changes in your code. Lluis provided a case, and it was obscure. That switch seems like a rare or non-existent use-case that should not guide the API.

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 5:15 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Fri, Jul 1, 2011 at 10:00 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Fri, Jul 1, 2011 at 4:34 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:50 AM, Matthew Brett

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Bruce Southey
On 07/01/2011 10:15 AM, Nathaniel Smith wrote: On Fri, Jul 1, 2011 at 7:09 AM, Mark Wiebemwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brettmatthew.br...@gmail.com wrote: Do you see problems with the alterNEP proposal? Yes, I really like my design as it stands now, and the

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 5:17 PM, Benjamin Root ben.r...@ou.edu wrote: On Fri, Jul 1, 2011 at 11:00 AM, Matthew Brett matthew.br...@gmail.com wrote: You can't switch between the two approaches without big changes in your code. Lluis provided a case, and it was obscure.  That switch

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Matthew Brett
Hi, On Fri, Jul 1, 2011 at 5:18 PM, Bruce Southey bsout...@gmail.com wrote: On 07/01/2011 10:15 AM, Nathaniel Smith wrote: I really find that you are 'splitting hairs' in your arguments as it really has to be up to the application on how missing values and NaN have to be handled. I see no

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Benjamin Root
On Fri, Jul 1, 2011 at 11:20 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Fri, Jul 1, 2011 at 5:17 PM, Benjamin Root ben.r...@ou.edu wrote: On Fri, Jul 1, 2011 at 11:00 AM, Matthew Brett matthew.br...@gmail.com wrote: You can't switch between the two approaches without

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Christopher Jordan-Squire
This is kind of late to be jumping into the 'long thread of doom', but I've been following most of the posts, so I'd figured I'd throw in my 2 cents. I'm Mark's officemate over the summer, and we've been talking daily about his design. I was skeptical of various details at first, but by now Mark's

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Mark Wiebe
On Fri, Jul 1, 2011 at 10:15 AM, Nathaniel Smith n...@pobox.com wrote: On Fri, Jul 1, 2011 at 7:09 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com wrote: Do you see problems with the alterNEP proposal? Yes, I really like my

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Lluís
Matthew Brett writes: Mainly: Reduced interoperability Meaning? You can't switch between the two approaches without big changes in your code. Lluis provided a case, and it was obscure. That switch seems like a rare or non-existent use-case that should not guide the API. The example

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Lluís
Nathaniel Smith writes: On Fri, Jul 1, 2011 at 7:09 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com wrote: Do you see problems with the alterNEP proposal? Yes, I really like my design as it stands now, and the alterNEP removes a

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Pierre GM
On Jul 1, 2011 7:14 PM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 10:15 AM, Nathaniel Smith n...@pobox.com wrote: On Fri, Jul 1, 2011 at 7:09 AM, Mark Wiebe mwwi...@gmail.com wrote: On Fri, Jul 1, 2011 at 6:58 AM, Matthew Brett matthew.br...@gmail.com wrote: Do you see

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Nathaniel Smith
On Fri, Jul 1, 2011 at 9:18 AM, Bruce Southey bsout...@gmail.com wrote: I am sorry that that is NOT true - DON'T just lump every one into this when they have clearly stated the opposite! Missing values are nothing special to me, just reality. There are many statistical applications where

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Nathaniel Smith
On Fri, Jul 1, 2011 at 9:29 AM, Benjamin Root ben.r...@ou.edu wrote: On Fri, Jul 1, 2011 at 11:20 AM, Matthew Brett matthew.br...@gmail.com wrote: On Fri, Jul 1, 2011 at 5:17 PM, Benjamin Root ben.r...@ou.edu wrote: For more complicated functions like pcolor() and contour(), the arrays

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Nathaniel Smith
On Fri, Jul 1, 2011 at 9:29 AM, Christopher Jordan-Squire cjord...@uw.edu wrote: This is kind of late to be jumping into the 'long thread of doom', but I've been following most of the posts, so I'd figured I'd throw in my 2 cents. I'm Mark's officemate over the summer, and we've been talking

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-07-01 Thread Eric Firing
On 07/01/2011 06:40 PM, Nathaniel Smith wrote: On Fri, Jul 1, 2011 at 9:29 AM, Christopher Jordan-Squire BTW, you can't access the memory of a masked value by taking a view, at least if I'm reading this version of the NEP correctly, and it seems to be the latest:

[Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Matthew Brett
Hi, On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith n...@pobox.com wrote: Anyway, it's pretty clear that in this particular case, there are two distinct features that different people want: the missing data feature, and the masked array feature. The more I think about it, the less I see how

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Pierre GM
On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: ### A alternative-NEP on masking and missing values ### I like the idea of two different special values, np.NA for missing values, np.IGNORE for masked

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Charles R Harris
On Thu, Jun 30, 2011 at 7:31 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith n...@pobox.com wrote: Anyway, it's pretty clear that in this particular case, there are two distinct features that different people want: the missing data

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Dag Sverre Seljebotn
On 06/30/2011 04:17 PM, Charles R Harris wrote: On Thu, Jun 30, 2011 at 7:31 AM, Matthew Brett matthew.br...@gmail.com mailto:matthew.br...@gmail.com wrote: Hi, On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith n...@pobox.com mailto:n...@pobox.com wrote: Anyway, it's

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Charles R Harris
On Thu, Jun 30, 2011 at 8:17 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Thu, Jun 30, 2011 at 7:31 AM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith n...@pobox.com wrote: Anyway, it's pretty clear that in this particular

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Matthew Brett
Hi, On Thu, Jun 30, 2011 at 2:58 PM, Pierre GM pgmdevl...@gmail.com wrote: On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: ### A alternative-NEP on masking and missing values ### I like the idea of

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Pierre GM
On Jun 30, 2011, at 5:38 PM, Matthew Brett wrote: Hi, On Thu, Jun 30, 2011 at 2:58 PM, Pierre GM pgmdevl...@gmail.com wrote: On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: ### A alternative-NEP on masking and missing values

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Matthew Brett
Hi, On Thu, Jun 30, 2011 at 5:03 PM, Pierre GM pgmdevl...@gmail.com wrote: On Jun 30, 2011, at 5:38 PM, Matthew Brett wrote: Hi, On Thu, Jun 30, 2011 at 2:58 PM, Pierre GM pgmdevl...@gmail.com wrote: On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote:

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Nathaniel Smith
On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett matthew.br...@gmail.com wrote: In the interest of making the discussion as concrete as possible, here is my draft of an alternative proposal for NAs and masking, based on Nathaniel's comments.  Writing it, it seemed to me that Nathaniel is right,

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Matthew Brett
Hi, On Thu, Jun 30, 2011 at 6:51 PM, Nathaniel Smith n...@pobox.com wrote: On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett matthew.br...@gmail.com wrote: In the interest of making the discussion as concrete as possible, here is my draft of an alternative proposal for NAs and masking, based on

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Lluís
Matthew Brett writes: [...] I'm afraid, like you, I'm a little lost in the world of masking, because I only need the NAs. I was trying to see if I could come up with an API that picked up some of the syntactic convenience of NAs, without conflating NAs with IGNOREs. I guess we need some

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Nathaniel Smith
On Thu, Jun 30, 2011 at 11:27 AM, Lluís xscr...@gmx.net wrote: As I tried to convey on my other mail, separating both will force you to either: * Make a copy of the array before passing it to another routine (because  the routine will assign np.NA but you still want the original data) To

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Matthew Brett
Hi, On Thu, Jun 30, 2011 at 7:27 PM, Lluís xscr...@gmx.net wrote: Matthew Brett writes: [...] I'm afraid, like you, I'm a little lost in the world of masking, because I only need the NAs.  I was trying to see if I could come up with an API that picked up some of the syntactic convenience of

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Lluís
Nathaniel Smith writes: On Thu, Jun 30, 2011 at 11:27 AM, Lluís xscr...@gmx.net wrote: As I tried to convey on my other mail, separating both will force you to either: * Make a copy of the array before passing it to another routine (because  the routine will assign np.NA but you still want

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Lluís
Matthew Brett writes: Hi, On Thu, Jun 30, 2011 at 7:27 PM, Lluís xscr...@gmx.net wrote: Matthew Brett writes: [...] I'm afraid, like you, I'm a little lost in the world of masking, because I only need the NAs.  I was trying to see if I could come up with an API that picked up some of the

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Matthew Brett
Hi, On Thu, Jun 30, 2011 at 9:01 PM, Lluís xscr...@gmx.net wrote: Matthew Brett writes: Hi, On Thu, Jun 30, 2011 at 7:27 PM, Lluís xscr...@gmx.net wrote: Matthew Brett writes: [...] I'm afraid, like you, I'm a little lost in the world of masking, because I only need the NAs.  I was trying

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Gary Strangman
It seems to me, that what ``func`` should do, if it wants you to be able to unmask the NAs, is to make a masked array view of ``arr``, and return that.   And indeed the simplicity of the separated API immediately makes that clear - in my view at least. I agree on this example. My only concern

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Charles R Harris
On Thu, Jun 30, 2011 at 6:02 PM, Matthew Brett matthew.br...@gmail.comwrote: Hi, On Thu, Jun 30, 2011 at 9:01 PM, Lluís xscr...@gmx.net wrote: Matthew Brett writes: Hi, On Thu, Jun 30, 2011 at 7:27 PM, Lluís xscr...@gmx.net wrote: Matthew Brett writes: [...] I'm afraid, like you,

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

2011-06-30 Thread Keith Goodman
On Thu, Jun 30, 2011 at 10:51 AM, Nathaniel Smith n...@pobox.com wrote: On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett matthew.br...@gmail.com wrote: In the interest of making the discussion as concrete as possible, here is my draft of an alternative proposal for NAs and masking, based on