Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On Mon, Apr 16, 2012 at 10:40:53PM -0500, Travis Oliphant wrote: The objectors object to any binary ABI change, but not specifically three pointers rather than two or one? Adding pointers is not really an ABI change (but removing them after they were there would be...) It's really just the addition of data to the NumPy array structure that they aren't going to use. Most of the time it would not be a real problem (the number of use-cases where you have a lot of small NumPy arrays is small), but when it is a problem it is very annoying. I think that something that the numpy community must be very careful about is ABI breakage. At the scale of a large and heavy institution, it is very costly. In my mind, this is the argument that should guide the discussion: does going one way of the other (removing NA or not) will lead us likely into ABI breakage ? My 2 cents, Gaël ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On Tue, Apr 17, 2012 at 6:44 AM, Travis Oliphant tra...@continuum.io wrote: Basically, there are two sets of changes as far as I understand right now: 1) ufunc infrastructure understands masked arrays 2) ndarray grew attributes to represent masked arrays I am proposing that we keep 1) but change 2) so that only certain kinds of NumPy arrays actually have the extra function pointers (effectively a sub-type). In essence, what I'm proposing is that the NumPy 1.6 PyArrayObject become a base-object, but the other members of the C-structure are not even present unless the Masked flag is set. Such changes would not require ripping code out --- just altering the presentation a bit. Yet, they could have large long-term implications, that we should explore before they get fixed. Whether masked arrays should be a formal sub-class is actually an un-related question and I generally lean in the direction of not encouraging sub-classes of the ndarray. The big questions are does this object work in the calculation infrastructure. Can I add an array to a masked array. Does it have a sum method? I think it could be argued that a masked array does have a is a relationship with an array. It can also be argued that it is better to have a has a relationship with an array and be-it's own-object. Either way, this object could still have it's first-part be binary compatible with a NumPy Array, and that is what I'm really suggesting. It sounds like the main implementation issue here is that this masked array class needs some way to coordinate with the ufunc infrastructure to efficiently and reliably handle the mask in calculations. The core ufunc code now knows how to handle masks, and this functionality is needed for where= and NA-dtypes, so obviously it's staying, independent of what we decide to do with masked arrays. So the question is just, how do we get the masked array and the ufuncs talking to each other so they can do the right thing. Perhaps we should focus, then, on how to create a better hooking mechanism for ufuncs? Something along these lines? http://mail.scipy.org/pipermail/numpy-discussion/2011-June/056945.html If done in a solid enough way, this would also solve other problems, e.g. we could make ufuncs work reliably on sparse matrices, which seems to trip people up on scipy-user every month or two. Of course, it's very tricky to get right :-( As far the masked array API: I'm still not convinced we know how we want these things to behave. The masked arrays in master currently implement MISSING semantics, but AFAICT everyone who wants MISSING semantics prefers NA-dtypes or even plain old NaN's over a masked implementation. And some of the current implementation's biggest backers, like Chuck, have argued that they should switch to skipNA=True, which is more of an IGNORED-style semantic. OTOH, there's still disagreement over how IGNORED-style semantics should even work (I'm thinking of that discussion about commutivity). The best existing model is numpy.ma -- but the numpy.ma API is quite different from the NEP, in more ways than just the default setting for skipNA. numpy.ma uses the opposite convention for mask values, it has additional concepts like the fillvalue, hardmask versus softmask, and then there's the whole way the NEP uses views to manage the mask. And I don't know which of these numpy.ma features are useful, which are extraneous, and which are currently useful but will become extraneous once the users who really wanted something more like NA-dtypes switch to those. So we all agree that masked arrays can be useful, and that numpy.ma has problems. But straightforwardly porting numpy.ma to C doesn't seem like it would help much, and neither does simply declaring that numpy.ma has been deprecated in favour of a new NEP-like API. So, I dunno. It seems like it might make the most sense to: 1) take the mask fields out of the core ndarray (while leaving the rest of Mark's infrastructure, as per above) 2) make sure we have the hooks needed so that numpy.ma, and NEP-like APIs, and whatever other experiments people want to try, can all integrate well with ufuncs, and make any other extensions that are generally useful and required so that they can work well 3) once we've experimented, move the winner into the core. Or whatever else makes sense to do once we understand what we're trying to accomplish. -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] f2py with int8
Thanks Sameer. I confirmed on my side as well. I will try to understand the why part now. Much appreciated. On Mon, Apr 16, 2012 at 11:58 PM, Sameer Grover sameer.grove...@gmail.comwrote: On Tuesday 17 April 2012 11:02 AM, John Mitchell wrote: Hi, I am using f2py to pass a numpy array of type numpy.int8 to fortran. It seems like I am misunderstanding something because I just can't make it work. Here is what I am doing. PYTHON b=numpy.array(numpy.zeros(shape=(10,),dtype=numpy.int8),order='F') b[0]=1 b[2]=1 b[3]=1 b array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0], dtype=int8) FORTRAN subroutine print_bit_array(bits,n) use iso_fortran_env integer,intent(in)::n integer(kind=int8),intent(in),dimension(n)::bits print*,'bits = ',bits end subroutine print_bit_array RESULT when calling fortran from python bits = 1000000010 Any Ideas? thanks, John ___ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion It seems to work if integer(kind=int8) is replaced with integer(8) or integer(1). Don't know why, though. Sameer ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: Mark and I will have conversations about NumPy while he is in Austin. There are many other active stake-holders whose opinions and views are essential for major changes. Mark and I are working on other things besides just NumPy and all NumPy changes will be discussed on list and require consensus or super-majority for NumPy itself to change. I'm not sure if that helps. Is there more we can do? As you might have heard me say before, my concern is that it has not been easy to have good discussions on this list. I think the problem has been that is has not been clear what the culture was, and how decisions got made, and that had led to some uncomfortable and unhelpful discussions. My plea would be for you as BDF$N to strongly encourage on-list discussions and discourage off-list discussions as far as possible, and to help us make the difficult public effort to bash out the arguments to clarity and consensus. I know that's a big ask. Hi Matthew, As you know, I agree with everything you just said :-). So in interest of transparency, I should add: I have been in touch with Travis some off-list, and the main topic has been how to proceed in a way that let's us achieve public consensus. -- Nathaniel ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] f2py with int8
Ah, come to think of it, I think that f2py only supports literal kind values. Maybe that's your problem. Paul On 17. apr. 2012, at 07:58, Sameer Grover wrote: On Tuesday 17 April 2012 11:02 AM, John Mitchell wrote: Hi, I am using f2py to pass a numpy array of type numpy.int8 to fortran. It seems like I am misunderstanding something because I just can't make it work. Here is what I am doing. PYTHON b=numpy.array(numpy.zeros(shape=(10,),dtype=numpy.int8),order='F') b[0]=1 b[2]=1 b[3]=1 b array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0], dtype=int8) FORTRAN subroutine print_bit_array(bits,n) use iso_fortran_env integer,intent(in)::n integer(kind=int8),intent(in),dimension(n)::bits print*,'bits = ',bits end subroutine print_bit_array RESULT when calling fortran from python bits = 1000000010 Any Ideas? thanks, John ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion It seems to work if integer(kind=int8) is replaced with integer(8) or integer(1). Don't know why, though. Sameer ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] f2py with int8
Thanks Paul. I suppose this is now going slightly out of bounds for f2py. What I am looking for is the fortran kind type for a byte. I thought that this was int8. I guess the question is how to identify the kind type. Although I have verified that integer(1) seems to work for me, I would really like to know why and your answer alludes to that. Please excuse my ignorance on this topic. Can you perhaps educate me a little on 'literal kind values'? I take you to mean that 'int8' is not a literal kind value while 1 and 8 are examples of literal kind values. Thanks, John On Tue, Apr 17, 2012 at 10:12 AM, Paul Anton Letnes paul.anton.let...@gmail.com wrote: Ah, come to think of it, I think that f2py only supports literal kind values. Maybe that's your problem. Paul On 17. apr. 2012, at 07:58, Sameer Grover wrote: On Tuesday 17 April 2012 11:02 AM, John Mitchell wrote: Hi, I am using f2py to pass a numpy array of type numpy.int8 to fortran. It seems like I am misunderstanding something because I just can't make it work. Here is what I am doing. PYTHON b=numpy.array(numpy.zeros(shape=(10,),dtype=numpy.int8),order='F') b[0]=1 b[2]=1 b[3]=1 b array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0], dtype=int8) FORTRAN subroutine print_bit_array(bits,n) use iso_fortran_env integer,intent(in)::n integer(kind=int8),intent(in),dimension(n)::bits print*,'bits = ',bits end subroutine print_bit_array RESULT when calling fortran from python bits = 1000000010 Any Ideas? thanks, John ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion It seems to work if integer(kind=int8) is replaced with integer(8) or integer(1). Don't know why, though. Sameer ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
Hi, On Tue, Apr 17, 2012 at 7:24 AM, Nathaniel Smith n...@pobox.com wrote: On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: Mark and I will have conversations about NumPy while he is in Austin. There are many other active stake-holders whose opinions and views are essential for major changes. Mark and I are working on other things besides just NumPy and all NumPy changes will be discussed on list and require consensus or super-majority for NumPy itself to change. I'm not sure if that helps. Is there more we can do? As you might have heard me say before, my concern is that it has not been easy to have good discussions on this list. I think the problem has been that is has not been clear what the culture was, and how decisions got made, and that had led to some uncomfortable and unhelpful discussions. My plea would be for you as BDF$N to strongly encourage on-list discussions and discourage off-list discussions as far as possible, and to help us make the difficult public effort to bash out the arguments to clarity and consensus. I know that's a big ask. Hi Matthew, As you know, I agree with everything you just said :-). So in interest of transparency, I should add: I have been in touch with Travis some off-list, and the main topic has been how to proceed in a way that let's us achieve public consensus. I'm glad to hear that discussion is happening, but please do have it on list. If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. So, yes, you might get a better outcome for this specific case, but a worse outcome in the long term, because the list will start to feel that it's for signing off or voting rather than discussion, and that - I feel sure - would lead to worse decisions. The other issue is that there's a reason you are having the discussion off-list - which is that it was getting difficult on-list. But - again - a personal view - that really has to be addressed directly by setting out the rules of engagement and modeling the kind of discussion we want to have. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On 04/17/2012 08:40 AM, Matthew Brett wrote: Hi, On Tue, Apr 17, 2012 at 7:24 AM, Nathaniel Smithn...@pobox.com wrote: On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brettmatthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphanttra...@continuum.io wrote: Mark and I will have conversations about NumPy while he is in Austin. There are many other active stake-holders whose opinions and views are essential for major changes.Mark and I are working on other things besides just NumPy and all NumPy changes will be discussed on list and require consensus or super-majority for NumPy itself to change. I'm not sure if that helps. Is there more we can do? As you might have heard me say before, my concern is that it has not been easy to have good discussions on this list. I think the problem has been that is has not been clear what the culture was, and how decisions got made, and that had led to some uncomfortable and unhelpful discussions. My plea would be for you as BDF$N to strongly encourage on-list discussions and discourage off-list discussions as far as possible, and to help us make the difficult public effort to bash out the arguments to clarity and consensus. I know that's a big ask. Hi Matthew, As you know, I agree with everything you just said :-). So in interest of transparency, I should add: I have been in touch with Travis some off-list, and the main topic has been how to proceed in a way that let's us achieve public consensus. ...when possible without paralysis. I'm glad to hear that discussion is happening, but please do have it on list. If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. So, yes, you might get a better outcome for this specific case, but a worse outcome in the long term, because the list will start to feel that it's for signing off or voting rather than discussion, and that - I feel sure - would lead to worse decisions. I think you are over-stating the case a bit. Taking what you say literally, one might conclude that numpy people should never meet and chat, or phone each other up and chat. But such small conversations are an important extension and facilitator of individual thinking. Major decisions do need to get hashed out publicly, but mailing list discussions are only one part of the thinking and decision process. Eric The other issue is that there's a reason you are having the discussion off-list - which is that it was getting difficult on-list. But - again - a personal view - that really has to be addressed directly by setting out the rules of engagement and modeling the kind of discussion we want to have. Cheers, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On Tue, Apr 17, 2012 at 11:40 AM, Matthew Brett matthew.br...@gmail.com wrote: I'm glad to hear that discussion is happening, but please do have it on list. If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. I'm afraid I have to disagree: you seem to be proposing an absolute, 'zero-tolerance'-style policy against any off-list discussion. The only thing ZT policies achieve is to remove common sense and human judgement from a process, invariably causing more harm than they do good, no matter how well intentioned. There are perfectly reasonable cases where a quick phone call may be a more effective and sensible way to work than an on-list discussion. The question isn't whether someone, somewhere, had an off-list discussion or not; it's whether *the main decision making process* is being handled transparently or not. I trust that Nathaniel and Travis had a sensible reason to speak off-list; as long as it appears clear that the *decisions about numpy* are being made via public discussion with room for all necessary input and confrontation of disparate viewpoints, I don't care what they talk about in private. In IPython, I am constantly fielding private emails that I very often refer to the list because they make more sense there, but I also have off-list discussions when I consider that to be the right thing to do. And I certainly hope nobody ever asks me to *never* have an off-list discussion. I try very hard to ensure that the direction of the project is very transparent, with redundant points (people) of access to critical resources and a good vetting of key decisions with public input (e.g. our first IPEP at https://github.com/ipython/ipython/issues/1611). If I am failing at that, I hope people will call me out *on that point*, but not on whether I ever pick up the phone or email to talk about IPython off-list. Let's try to trust for one minute that the actual decisions will be made here with solid debate and project-wide input, and seek change only if we have evidence that this isn't happening (not evidence of a meta-problem that isn't a problem here). Best, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
Hi, On Tue, Apr 17, 2012 at 12:04 PM, Fernando Perez fperez@gmail.com wrote: On Tue, Apr 17, 2012 at 11:40 AM, Matthew Brett matthew.br...@gmail.com wrote: I'm glad to hear that discussion is happening, but please do have it on list. If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. I'm afraid I have to disagree: you seem to be proposing an absolute, 'zero-tolerance'-style policy against any off-list discussion. The only thing ZT policies achieve is to remove common sense and human judgement from a process, invariably causing more harm than they do good, no matter how well intentioned. Right - but that would be an absurd overstatement of what I said. There's no point in addressing something I didn't say and no sensible person would think. Indeed, it makes the discussion harder. It's just exhausting to have to keep stating the obvious. Of course discussions happen off-list. Of course sometimes that has to happen. Of course that can be a better and quicker way of having discussions. However, in this case the Let's try to trust for one minute that the actual decisions will be made here with solid debate and project-wide input, and seek change only if we have evidence that this isn't happening (not evidence of a meta-problem that isn't a problem here). meta-problem that is a real problem is that we've shown ourselves that we are not currently good at having discussions on list. There are clearly reasons for that, and also clearly, they can be addressed. The particular point I am making is neither silly nor extreme nor vapid. It is simply that, in order to make discussion work better on the list, it is in my view better to make an explicit effort to make the discussions - explicit. Yours in Bay Area opposition, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On Tue, Apr 17, 2012 at 12:10 PM, Matthew Brett matthew.br...@gmail.com wrote: Right - but that would be an absurd overstatement of what I said. There's no point in addressing something I didn't say and no sensible person would think. Indeed, it makes the discussion harder. Well, in that case neither Eric Firing nor I are 'sensible persons', since that's how we both understood what you said (Eric's email appeared to me as a more concise/better phrased version of the same points I was making). You said: I'm glad to hear that discussion is happening, but please do have it on list. If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. I don't think it's an 'absurd overstatement' to interpret that as don't have discussions off-list, but hey, it may just be me :) meta-problem that is a real problem is that we've shown ourselves that we are not currently good at having discussions on list. Oh, I know that did happen in the past regarding this very topic (the big NA mess last summer); what I meant was to try and trust that *this time around* things might be already moving in a better direction, which it seems to me they are. It seems to me that everyone is genuinely trying to tackle the discussion/consensus questions head-on right on the list, and that's why I proposed waiting to see if there were really any problems before asking Nathaniel not to have any discussion off-list (esp. since we have no evidence that what they talked about had any impact on any decisions bypassing the open forum). Best, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
On Tue, Apr 17, 2012 at 12:32 PM, Fernando Perez fperez@gmail.com wrote: On Tue, Apr 17, 2012 at 12:10 PM, Matthew Brett matthew.br...@gmail.com wrote: Right - but that would be an absurd overstatement of what I said. There's no point in addressing something I didn't say and no sensible person would think. Indeed, it makes the discussion harder. Well, in that case neither Eric Firing nor I are 'sensible persons', since that's how we both understood what you said (Eric's email appeared to me as a more concise/better phrased version of the same points I was making). You said: I'm glad to hear that discussion is happening, but please do have it on list. If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. I don't think it's an 'absurd overstatement' to interpret that as don't have discussions off-list, but hey, it may just be me :) The absurd over-statement is the following: I'm afraid I have to disagree: you seem to be proposing an absolute, 'zero-tolerance'-style policy against any off-list discussion. meta-problem that is a real problem is that we've shown ourselves that we are not currently good at having discussions on list. Oh, I know that did happen in the past regarding this very topic (the big NA mess last summer); what I meant was to try and trust that *this time around* things might be already moving in a better direction, which it seems to me they are. It seems to me that everyone is genuinely trying to tackle the discussion/consensus questions head-on right on the list, and that's why I proposed waiting to see if there were really any problems before asking Nathaniel not to have any discussion off-list (esp. since we have no evidence that what they talked about had any impact on any decisions bypassing the open forum). The question - which seems to me to be sensible rational and important - is how to get better at on-list discussion, and whether taking this particular discussion mainly off-list is good or bad in that respect. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)
I have never found mailing lists good places for discussion and consensus. I think the format itself does not lend itself to involvement, carefully considered (or the ability to change) positions, or voting since all of it can be so easily lost within all of the quoting, the back and forth, people walking away,,,etc. And you also want involvement from people who don't have x hours to craft a finely worded, politically correct, and detailed response. I am not advocating this particular system, but something like http://meta.programmers.stackexchange.com/ would be a better platform for building to a decision when there are many choices to be made. Now about ma, NA, missing... I am just an engineer working in water resources and I had lots of difficulty reading the NEP (so slepy) so I will be the first to admit that I probably have something wrong. Just for reference (since I missed it the first time around) Nathaniel posted this page at https://github.com/njsmith/numpy/wiki/NA-discussion-status I think that I could adapt to everything that is discussed in the NEP, but I do have some comments about things that puzzled me. I don't need answers, but if I am puzzled maybe others are also. First - 'maskna=True'? Tested on development version of numpy... a = np.arange(10, maskna = True) a[:2] = np.NA a array([NA, NA, 2, 3, 4, 5, 6, 7, 8, 9]) Why do I have to specify 'maskna = True'? If NA and ndarray are intended to be combined in some way, then I don't think that I need this. During development, I understand, but the NEP shouldn't have it. Heck, even if you keep NA and ndarrays separate when someone tries to set a ndarray element with np.NA, instead of a ValueError convert to an NA array. I say that very casually as if I know how to do it. I do have a proof, but the margin is too small to include it. :-) I am torn about 'skipna=True'. I think I understand the desire for explicit behavior, but I suspect that every operation that I would use a NA array for, would require 'skipna=True'. Actually, I don't use that many reducing functions, so maybe not a big deal. Regardless of the skipna setting, a related idea that could be useful for reducing functions is to set an 'includesna' attribute with the returned scalar value. The view() didn't work as described in the NEP, where np.NA isn't propagated back to the original array. This could be because the NEP references a 'missingdata' work in progress branch and I don't know what has been merged. I can force the NEP described behavior if I set 'd.flags.ownmaskna=True'. d = a.view() d array([NA, NA, 2, 3, 4, 5, 6, 7, 8, 9]) d[0] = 4 a array([4, NA, 2, 3, 4, 5, 6, 7, 8, 9]) d array([4, NA, 2, 3, 4, 5, 6, 7, 8, 9]) d[6] = np.NA d array([4, NA, 2, 3, 4, 5, NA, 7, 8, 9]) a array([4, NA, 2, 3, 4, 5, NA, 7, 8, 9]) In the NEP 'Accessing a Boolean Mask' section there is a comment about... actually I don't understand this section at all. Especially about a boolean byle level mask? Why would it need to be a byte level mask in order to be viewed? The logic also of mask = True or False, that can be easily handled by using a better name for the flag. 'mask = True' means that the value is masked (missing), where if 'exposed = True' is used that means the value is not masked (not missing). The biggest question mark to me is that 'a[0] = np.NA' is destructive and (using numpy.ma) 'a.mask[0] = True' is not. Is that a big deal? I am trying to think back on all of my 'ma' code and try to remember if I masked, then unmasked values and I don't recall any time that I did that. Of course my use cases are constrained to what I have done in the past. It feels like a bad idea, for the sake of saving the memory for the mask bits. Now, the amazing thing is that understanding so little, doing even less of the work, I get to vote. Sounds like America! I would really like to see NA in the wild, and I think that I can adapt my ma code to it, so +1. If it has to wait until 1.8, +1. If it has to wait until 1.9, +1. Kindest regards, Tim ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] [Meta] Was: Re: Removing masked arrays for 1.7? (Was 1.7 blockers)
[ Making a separate thread so the NA one can stay on topic, since I haven't actually followed the discussion well enough to contribute on the technical points ] On Tue, Apr 17, 2012 at 12:43 PM, Matthew Brett matthew.br...@gmail.com wrote: The absurd over-statement is the following: I'm afraid I have to disagree: you seem to be proposing an absolute, 'zero-tolerance'-style policy against any off-list discussion. Well, that's how I understood what you said; sorry if I mis-interpreted it, but that is indeed how I read your post. The question - which seems to me to be sensible rational and important - is how to get better at on-list discussion, and whether taking this particular discussion mainly off-list is good or bad in that respect. And on that we completely agree. I just think that in this case, we can try to give people a chance and see if things go better: things have changed since last summer and there's encouraging evidence to suggest that a solid, open discussion will take place this time around. I just propose we give that process a chance first... Best, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] All of the PyData videos are now up at the Marakana site
Hi folks, A number of you expressed interest in attending the PyData workshop last month and unfortunately we had very tight space restrictions. But thanks to the team at Marakana, who pitched in and were willing to film, edit and post videos for many of the talks, you can access them all here: http://marakana.com/s/2012_pydata_workshop,1090/index.html They are in 720p so you can actually read the terminals, though I think you have to click the YouTube link to be able to change the resolution. Enjoy! f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
