2011/10/28 Stéfan van der Walt
> On Fri, Oct 28, 2011 at 4:19 PM, Charles R Harris
> wrote:
> > Memory use is a known problem. One way to start addressing it might be to
> > implement a "bit" arraytype. It might even be possible to prototype that
> on
> > top of the existing types. Views make bi
On Fri, Oct 28, 2011 at 4:19 PM, Charles R Harris
wrote:
> Memory use is a known problem. One way to start addressing it might be to
> implement a "bit" arraytype. It might even be possible to prototype that on
> top of the existing types. Views make bit arrays a bit more interesting ;)
Since 1/8
On Fri, Oct 28, 2011 at 5:05 PM, Chris.Barker wrote:
> On 10/28/11 11:37 AM, Matthew Brett wrote:
> > The main motivation for the alterNEP was our strong feeling that
> > separating ABSENT and IGNORE was easier to comprehend and cleaner.
>
> I don't know about easier to comprehend, or cleaner, bu
On 10/28/11 11:37 AM, Matthew Brett wrote:
> The main motivation for the alterNEP was our strong feeling that
> separating ABSENT and IGNORE was easier to comprehend and cleaner.
I don't know about easier to comprehend, or cleaner, but it is more
feature-full.
I see two issues here:
1) being ab
Hi,
On Fri, Oct 28, 2011 at 1:52 PM, Benjamin Root wrote:
>
>
> On Fri, Oct 28, 2011 at 3:22 PM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> On Fri, Oct 28, 2011 at 1:14 PM, Benjamin Root wrote:
>> >
>> >
>> > On Fri, Oct 28, 2011 at 3:02 PM, Matthew Brett
>> > wrote:
>> >>
>> >> You and I know tha
On Fri, Oct 28, 2011 at 3:22 PM, Matthew Brett wrote:
> Hi,
>
> On Fri, Oct 28, 2011 at 1:14 PM, Benjamin Root wrote:
> >
> >
> > On Fri, Oct 28, 2011 at 3:02 PM, Matthew Brett
> > wrote:
> >>
> >> You and I know that I've got an array with values [99, 100, 3] and a
> >> mask with values [False,
2011/10/28 Stéfan van der Walt
> On Fri, Oct 28, 2011 at 12:47 PM, Benjamin Root wrote:
> >
> > 2011/10/28 Stéfan van der Walt
> >> The
> >> implementation as it stands essentially gives us a faster and more
> >> integrated version of numpy.ma; but it has become clear from this
> >> conversatio
Hi,
On Fri, Oct 28, 2011 at 1:14 PM, Benjamin Root wrote:
>
>
> On Fri, Oct 28, 2011 at 3:02 PM, Matthew Brett
> wrote:
>>
>> You and I know that I've got an array with values [99, 100, 3] and a
>> mask with values [False, False, True]. So maybe I'd like to see what
>> happens if I take off the
On Fri, Oct 28, 2011 at 1:14 PM, Benjamin Root wrote:
> Considering that you have admitted before to not regularly using masked
> arrays, I seriously doubt that you would be able to judge whether this is a
> significant detriment or not.
Let's not be unreasonable; Matthew has a valid concern (may
On Fri, Oct 28, 2011 at 3:02 PM, Matthew Brett wrote:
>
> You and I know that I've got an array with values [99, 100, 3] and a
> mask with values [False, False, True]. So maybe I'd like to see what
> happens if I take off the mask from the second value. I know that's
> what I want to do, but I
On Fri, Oct 28, 2011 at 12:47 PM, Benjamin Root wrote:
>
> 2011/10/28 Stéfan van der Walt
>> The
>> implementation as it stands essentially gives us a faster and more
>> integrated version of numpy.ma; but it has become clear from this
>> conversation that such an approach overlooks a very common
Hi,
On Fri, Oct 28, 2011 at 12:15 PM, Lluís wrote:
> Summarizing: let's forget for a moment that "mask" has a meaning in english:
This is at the core of the problem. You and I know what's really
going on - there's a mask over the data. But in what follows we're
going to try and pretend that
2011/10/28 Stéfan van der Walt
> On Fri, Oct 28, 2011 at 11:16 AM, Benjamin Root wrote:
> > this by making missing data front-and-center. However, my belief is that
> > Mark's approach is easier to comprehend and is cleaner. Cleaner features
> > means that it is more likely to be used.
>
> Cle
On Fri, Oct 28, 2011 at 11:16 AM, Benjamin Root wrote:
> this by making missing data front-and-center. However, my belief is that
> Mark's approach is easier to comprehend and is cleaner. Cleaner features
> means that it is more likely to be used.
Cleaner features may be easier to adopt, but wh
I haven't actually tested the code, but AFAIK the following is a short overview
with examples of how the two orthogonal feature axis (ABSENT/IGNORE and
PROPAGATE/SKIP) are related and how it all is supposed to work.
I have never talked to Mark or anybody else in this list (that is, outside of
this
Hi,
On Fri, Oct 28, 2011 at 11:16 AM, Benjamin Root wrote:
> On Fri, Oct 28, 2011 at 12:39 PM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> On Thu, Oct 27, 2011 at 10:56 PM, Benjamin Root wrote:
>> >
>> >
>> > On Thursday, October 27, 2011, Charles R Harris
>> >
>> > wrote:
>> >>
>> >>
>> >> On Thu,
On Fri, Oct 28, 2011 at 12:58 PM, Gary Strangman wrote:
>
> >> I wonder if that might be handled as a scikits-image extension, rather
> >> than core numpy?
> >
> > I think Stefan and Nathaniel and Gary Strangman and others are saying
> > we don't want to pay the price of a large memory hike for m
On Fri, Oct 28, 2011 at 12:39 PM, Matthew Brett wrote:
> Hi,
>
> On Thu, Oct 27, 2011 at 10:56 PM, Benjamin Root wrote:
> >
> >
> > On Thursday, October 27, 2011, Charles R Harris <
> charlesr.har...@gmail.com>
> > wrote:
> >>
> >>
> >> On Thu, Oct 27, 2011 at 7:16 PM, Travis Oliphant <
> oliph..
>> I wonder if that might be handled as a scikits-image extension, rather
>> than core numpy?
>
> I think Stefan and Nathaniel and Gary Strangman and others are saying
> we don't want to pay the price of a large memory hike for masking. I
> suspect that Nathaniel is right, and that a large major
Hi,
On Fri, Oct 28, 2011 at 9:21 AM, Chris.Barker wrote:
> On 10/27/11 7:51 PM, Travis Oliphant wrote:
>> As I mentioned. I find the ability to separate an ABSENT idea from an
>> IGNORED idea convincing. In other words, I think distinguishing between
>> masks and bit-patterns is not just an imple
Hi,
On Thu, Oct 27, 2011 at 10:56 PM, Benjamin Root wrote:
>
>
> On Thursday, October 27, 2011, Charles R Harris
> wrote:
>>
>>
>> On Thu, Oct 27, 2011 at 7:16 PM, Travis Oliphant
>> wrote:
>>>
>>> That is a pretty good explanation. I find myself convinced by Matthew's
>>> arguments. I thi
On 10/27/11 7:51 PM, Travis Oliphant wrote:
> As I mentioned. I find the ability to separate an ABSENT idea from an
> IGNORED idea convincing. In other words, I think distinguishing between
> masks and bit-patterns is not just an implementation detail, but
> provides a useful concept for multiple u
On Thu, Oct 27, 2011 at 8:51 PM, Travis Oliphant wrote:
> As I mentioned. I find the ability to separate an ABSENT idea from an
> IGNORED idea convincing.In other words, I think distinguishing between
> masks and bit-patterns is not just an implementation detail, but provides a
> useful conc
On Fri, Oct 28, 2011 at 10:45:09AM +0200, Han Genuit wrote:
> Also, I like the short and concise abbreviation for 'Not Applicable',
> NA. It has more common uses than IGNORE.
> (See also here:
> http://www.johndcook.com/R_language_for_programmers.html#missing)
That's a very R centric point a view:
Hi all,
On Thu, Oct 27, 2011 at 7:51 PM, Travis Oliphant wrote:
> I understand exactly what it would take to add bit-patterns to NumPy. I
> also understand what Mark did and agree that it is possible to add Matthew's
> idea to the current code-base. I think it is worth exploring
Another data p
Yes, to further iterate on that, you can also create multiple masked
views with each its own mask properties. It would be ambiguous to mix
a bit-pattern NA together with standard NA's in the same mask, but you
can make different specialized masked views on the same data.
Also, I like the short and
> It should be possible to remove a mask when copying an array.
>
This was a concession on the part of those pushing for masks. Eventually, I
ended up realizing that it resulted in a stronger design.
Consider the following:
foo(a[4:10])
Should function foo be able to access the rest of array "
There is a way to assign whole masks in the current implementation:
>>> a = np.arange(9, maskna=True).reshape((3,3))
>>> a
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
>>> mask = np.array([[False, False, True],
[False, True, False],
On Thursday, October 27, 2011, Charles R Harris
wrote:
>
>
> On Thu, Oct 27, 2011 at 7:16 PM, Travis Oliphant
wrote:
>>
>> That is a pretty good explanation. I find myself convinced by Matthew's
arguments.I think that being able to separate ABSENT from IGNORED is a
good idea. I also like
As I mentioned. I find the ability to separate an ABSENT idea from an IGNORED
idea convincing.In other words, I think distinguishing between masks and
bit-patterns is not just an implementation detail, but provides a useful
concept for multiple use-cases.
I understand exactly what it woul
On Thu, Oct 27, 2011 at 7:16 PM, Travis Oliphant wrote:
> That is a pretty good explanation. I find myself convinced by Matthew's
> arguments.I think that being able to separate ABSENT from IGNORED is a
> good idea. I also like being able to control SKIP and PROPAGATE (but I
> think the cu
That is a pretty good explanation. I find myself convinced by Matthew's
arguments.I think that being able to separate ABSENT from IGNORED is a good
idea. I also like being able to control SKIP and PROPAGATE (but I think the
current implementation allows this already).
What is the count
Hi,
On Tue, Oct 25, 2011 at 7:56 PM, Travis Oliphant wrote:
> So, I am very interested in making sure I remember the details of the
> counterproposal. What I recall is that you wanted to be able to
> differentiate between a "bit-pattern" mask and a boolean-array mask in the
> API. I belie
Hi,
On Tue, Oct 25, 2011 at 7:56 PM, Travis Oliphant wrote:
> So, I am very interested in making sure I remember the details of the
> counterproposal. What I recall is that you wanted to be able to
> differentiate between a "bit-pattern" mask and a boolean-array mask in the
> API. I belie
There is also:
Missing/accumulating data
http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057406.html
An NA compromise idea -- many-NA
http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057408.html
NEPaNEP lessons - was: alterNEP
http://mail.scipy.org/pipermail/numpy-discussion
On 10/25/2011 04:56 PM, Travis Oliphant wrote:
> So, I am very interested in making sure I remember the details of the
> counterproposal.What I recall is that you wanted to be able to
> differentiate between a "bit-pattern" mask and a boolean-array mask
> in the API. I believe currently even
So, I am very interested in making sure I remember the details of the
counterproposal.What I recall is that you wanted to be able to
differentiate between a "bit-pattern" mask and a boolean-array mask in the API.
I believe currently even when bit-pattern masks are implemented the
differen
Hi,
Thank you for your gracious email.
On Tue, Oct 25, 2011 at 2:56 PM, Travis Oliphant wrote:
> It is a shame that Nathaniel and perhaps Matthew do not feel like their
> voice was heard. I wish I could have participated more fully in some of
> the discussions. I don't know if I could have re
It is a shame that Nathaniel and perhaps Matthew do not feel like their voice
was heard. I wish I could have participated more fully in some of the
discussions. I don't know if I could have really helped, but I would have
liked to have tried to perhaps work alongside Mark to integrate some of
Matthew Brett writes:
[...]
>>> If we do value constructive disagreement then we'll go out of our way
>>> to talk through the points of contention, and make sure that the
>>> people who disagree, especially the minority, feel that they have been
>>> fully heard.
>>>
>>> If we don't value construct
Hi,
On Tue, Oct 25, 2011 at 11:24 AM, Benjamin Root wrote:
> On Tue, Oct 25, 2011 at 1:03 PM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote:
>> > Matthew Brett writes:
>> >> I'm afraid I find this whole thread very unpleasant.
>> >
>> >> I have the odd impr
On Tue, Oct 25, 2011 at 1:03 PM, Matthew Brett wrote:
> Hi,
>
> On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote:
> > Matthew Brett writes:
> >> I'm afraid I find this whole thread very unpleasant.
> >
> >> I have the odd impression of being back at high school. Some of the
> >> big kids are pushing
Hi,
On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote:
> Matthew Brett writes:
>> I'm afraid I find this whole thread very unpleasant.
>
>> I have the odd impression of being back at high school. Some of the
>> big kids are pushing me around and then the other kids join in.
>
>> It didn't have to be
Matthew Brett writes:
> I'm afraid I find this whole thread very unpleasant.
> I have the odd impression of being back at high school. Some of the
> big kids are pushing me around and then the other kids join in.
> It didn't have to be this way.
> Someone could have replied like this to Nathani
Hi,
On Mon, Oct 24, 2011 at 11:44 PM, Han Genuit wrote:
> Well, if I may have a say, I think that an open source project is
> especially open when users as developers can contribute to the code
> base and can participate in discussions on how to improve the existing
> designs and ideas. I do not
Well, if I may have a say, I think that an open source project is
especially open when users as developers can contribute to the code
base and can participate in discussions on how to improve the existing
designs and ideas. I do not think a project is open when it crumbles
down into politics.. I ha
Charles R Harris writes:
[...]
> It might useful to have a way of setting global defaults, or something like a
> with statement. These are the sort of things that can be adjusted based on
> experience. For instance, I'm thinking skipna=1 is the natural default for the
> masked arrays.
I already ra
On Mon, Oct 24, 2011 at 11:12 AM, Wes McKinney wrote:
> On Mon, Oct 24, 2011 at 10:54 AM, Charles R Harris
> wrote:
> >
> >
> > On Mon, Oct 24, 2011 at 8:40 AM, Charles R Harris
> > wrote:
> >>
> >>
> >> On Sun, Oct 23, 2011 at 11:23 PM, Wes McKinney
> >> wrote:
> >>>
> >>> On Sun, Oct 23, 201
On Mon, Oct 24, 2011 at 10:54 AM, Charles R Harris
wrote:
>
>
> On Mon, Oct 24, 2011 at 8:40 AM, Charles R Harris
> wrote:
>>
>>
>> On Sun, Oct 23, 2011 at 11:23 PM, Wes McKinney
>> wrote:
>>>
>>> On Sun, Oct 23, 2011 at 8:07 PM, Eric Firing wrote:
>>> > On 10/23/2011 12:34 PM, Nathaniel Smith
24.10.2011 16:40, Charles R Harris kirjoitti:
[clip]
> The missing data functionality looks far more like R than numpy.ma
... and masked arrays must be explicitly requested by the user [1].
The MA stuff can "leak through" only if the user makes use of a library
that returns masked results (or exp
On Mon, Oct 24, 2011 at 8:40 AM, Charles R Harris wrote:
>
>
> On Sun, Oct 23, 2011 at 11:23 PM, Wes McKinney wrote:
>
>> On Sun, Oct 23, 2011 at 8:07 PM, Eric Firing wrote:
>> > On 10/23/2011 12:34 PM, Nathaniel Smith wrote:
>> >
>> >> like. And in this case I do think we can come up with an AP
On Sun, Oct 23, 2011 at 11:23 PM, Wes McKinney wrote:
> On Sun, Oct 23, 2011 at 8:07 PM, Eric Firing wrote:
> > On 10/23/2011 12:34 PM, Nathaniel Smith wrote:
> >
> >> like. And in this case I do think we can come up with an API that will
> >> make everyone happy, but that Mark's current API pro
Nathaniel Smith writes:
[...]
> Is the idea to continue the discussion and rework the API while it is in
> master, delaying the next release for as long as it takes to achieve
> consensus?
Well, for those who missed it, I think the first thing to do should be to
carefully read and discuss the cont
On Sun, Oct 23, 2011 at 8:07 PM, Eric Firing wrote:
> On 10/23/2011 12:34 PM, Nathaniel Smith wrote:
>
>> like. And in this case I do think we can come up with an API that will
>> make everyone happy, but that Mark's current API probably can't be
>> incrementally evolved to become that API.)
>>
>
On 10/23/2011 12:34 PM, Nathaniel Smith wrote:
> like. And in this case I do think we can come up with an API that will
> make everyone happy, but that Mark's current API probably can't be
> incrementally evolved to become that API.)
>
No one could object to coming up with an API that makes every
On Sun, Oct 23, 2011 at 2:29 PM, Eric Firing wrote:
> Ultimately, though, the numpy core developers must decide what goes in
> and what does not. Consensus is desirable but may not always be
> possible or optimal, especially if "consensus" is interpreted as
> "unanimity". There is a risk in deci
On Sun, Oct 23, 2011 at 3:28 PM, Benjamin Root wrote:
>
>
> On Sunday, October 23, 2011, Nathaniel Smith wrote:
> > On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
> > wrote:
> >> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett <
> matthew.br...@gmail.com>
> >> wrote:
> >>> I think this emai
On 10/23/2011 10:49 AM, Nathaniel Smith wrote:
> But I (and presumably others) were unaware of the pull request,
> because it turns out that actually Mark did*not* point to the pull
> request, at least in email to either me or numpy-discussion. As far as
> I can tell, the first time that pull requ
On Sunday, October 23, 2011, Nathaniel Smith wrote:
> On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
> wrote:
>> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett
>> wrote:
>>> I think this email might be a plea to the numpy steering group, and to
>>> Travis in particular, to see if we can use
On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
wrote:
> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett
> wrote:
>> I think this email might be a plea to the numpy steering group, and to
>> Travis in particular, to see if we can use a discussion of this series
>> of events to decide on a good
On 10/23/2011 04:07 PM, Robert Kern wrote:
> On Sun, Oct 23, 2011 at 20:58, Matthew Brett wrote:
>> Hi,
>>
>> On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
>> wrote:
>>>
>>>
>>> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett
>>> wrote:
Hi,
On Sun, Oct 23, 2011 at 11:21
Hi,
On Sun, Oct 23, 2011 at 1:07 PM, Robert Kern wrote:
> On Sun, Oct 23, 2011 at 20:58, Matthew Brett wrote:
>> Hi,
>>
>> On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
>> wrote:
>>>
>>>
>>> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett
>>> wrote:
Hi,
On Sun, Oct 23,
On Sun, Oct 23, 2011 at 20:58, Matthew Brett wrote:
> Hi,
>
> On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
> wrote:
>>
>>
>> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett
>> wrote:
>>>
>>> Hi,
>>>
>>> On Sun, Oct 23, 2011 at 11:21 AM, Nathaniel Smith wrote:
>>> > Hi all,
>>> >
>>> > I w
On Sun, Oct 23, 2011 at 12:21 PM, Nathaniel Smith wrote:
> Hi all,
>
> I was surprised today to notice that Mark's NA mask support appears to
> have been merged into numpy master and is described in the draft
> release notes[1]. My surprise is because merging it to mainline
> without any discussi
Hi,
On Sun, Oct 23, 2011 at 12:53 PM, Charles R Harris
wrote:
>
>
> On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> On Sun, Oct 23, 2011 at 11:21 AM, Nathaniel Smith wrote:
>> > Hi all,
>> >
>> > I was surprised today to notice that Mark's NA mask support appears to
>>
On Sun, Oct 23, 2011 at 12:54 PM, Matthew Brett wrote:
> Hi,
>
> On Sun, Oct 23, 2011 at 11:21 AM, Nathaniel Smith wrote:
> > Hi all,
> >
> > I was surprised today to notice that Mark's NA mask support appears to
> > have been merged into numpy master and is described in the draft
> > release not
Hi,
On Sun, Oct 23, 2011 at 11:21 AM, Nathaniel Smith wrote:
> Hi all,
>
> I was surprised today to notice that Mark's NA mask support appears to
> have been merged into numpy master and is described in the draft
> release notes[1]. My surprise is because merging it to mainline
> without any disc
23.10.2011 20:21, Nathaniel Smith kirjoitti:
> I was surprised today to notice that Mark's NA mask support appears to
> have been merged into numpy master and is described in the draft
> release notes[1]. My surprise is because merging it to mainline
> without any discussion on the list seems to co
Hi all,
I was surprised today to notice that Mark's NA mask support appears to
have been merged into numpy master and is described in the draft
release notes[1]. My surprise is because merging it to mainline
without any discussion on the list seems to contradict what what
Travis wrote in July, tha
69 matches
Mail list logo
