Re: [Numpy-discussion] NA masks in the next numpy release?
There is also: Missing/accumulating data http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057406.html An NA compromise idea -- many-NA http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057408.html NEPaNEP lessons - was: alterNEP http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057435.html NA/Missing Data Conference Call Summary http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057474.html HPC missing data - was: NA/Missing Data Conference Call Summary http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057482.html using the same vocabulary for missing value ideas http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057485.html towards a more productive missing values/masked arrays discussion... http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057511.html miniNEP1: where= argument for ufuncs http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057513.html miniNEP 2: NA support via special dtypes http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057542.html Missing Data development plan http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057567.html Missing Values Discussion http://mail.scipy.org/pipermail/numpy-discussion/2011-July/057579.html NA masks for NumPy are ready to test http://mail.scipy.org/pipermail/numpy-discussion/2011-August/058103.html ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
On 10/25/2011 04:56 PM, Travis Oliphant wrote: > So, I am very interested in making sure I remember the details of the > counterproposal.What I recall is that you wanted to be able to > differentiate between a "bit-pattern" mask and a boolean-array mask > in the API. I believe currently even when bit-pattern masks are > implemented the difference will be "hidden" from the user on the > Python level. > > I am sure to be missing other parts of the discussion as I have been > in and out of it. > > Thanks, > > -Travis The alternative-NEP is here: https://gist.github.com/1056379/ One thread of discussion is here: http://www.mail-archive.com/numpy-discussion@scipy.org/msg32268.html and continued here: http://www.mail-archive.com/numpy-discussion@scipy.org/msg32371.html Eric ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
So, I am very interested in making sure I remember the details of the counterproposal.What I recall is that you wanted to be able to differentiate between a "bit-pattern" mask and a boolean-array mask in the API. I believe currently even when bit-pattern masks are implemented the difference will be "hidden" from the user on the Python level. I am sure to be missing other parts of the discussion as I have been in and out of it. Thanks, -Travis On Oct 25, 2011, at 7:02 PM, Matthew Brett wrote: > Hi, > > Thank you for your gracious email. > > On Tue, Oct 25, 2011 at 2:56 PM, Travis Oliphant > wrote: >> It is a shame that Nathaniel and perhaps Matthew do not feel like their >> voice was heard. I wish I could have participated more fully in some of >> the discussions. I don't know if I could have really helped, but I would >> have liked to have tried to perhaps work alongside Mark to integrate some of >> the other ideas that had been expressed during the discussion. >> Unfortunately, I was traveling in NYC most of the time that Mark was >> working on this project and did not get a chance to interact with him as >> much as I would have liked. >> My view is that we didn't get quite to where I thought we would get, nor >> where I think we could be. I think Nathaniel and Matthew provided very >> specific feedback that was helpful in understanding other perspectives of a >> difficult problem. In particular, I really wanted bit-patterns >> implemented.However, I also understand that Mark did quite a bit of work >> and altered his original designs quite a bit in response to community >> feedback. I wasn't a major part of the pull request discussion, nor did I >> merge the changes, but I support Charles if he reviewed the code and felt >> like it was the right thing to do. I likely would have done the same thing >> rather than let Mark Wiebe's work languish. >> Merging Mark's code does not mean there is not more work to be done, but it >> is consistent with the reality that currently development on NumPy happens >> when people have the time to do it.I have not seen anything to convince >> me that there is not still time to make specific API changes that address >> some of the concerns. >> Perhaps, Nathaniel and or Matthew could summarize their concerns again and >> if desired submit a pull request to revert the changes. However, there is >> a definite bias against removing working code unless the arguments are very >> strong and receive a lot of support from others. > > Honestly - I am not sure whether there is any interest now, in the > arguments we made before. If there is, who is interested? I mean, > past politeness. > > I wasn't trying to restart that discussion, because I didn't know what > good it could do. At first I was hoping that we could ask whether > there was a better way of dealing with disagreements like this. > Later it seemed to me that the atmosphere was getting bad, and I > wanted to say that because I thought it was important. > >> Thank you for continuing to voice your opinions even when it may feel that >> the tide is against you. My view is that we only learn from people who >> disagree with us. > > Thank you for saying that. I hope that y'all will tell me if I am > making it harder for you to disagree, and I am sorry if I did so > here. > > Best, > > Matthew > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion --- Travis Oliphant Enthought, Inc. oliph...@enthought.com 1-512-536-1057 http://www.enthought.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] skip lines at the end of file with loadtxt
Many thanks Oliver!
i missed it in the description,
works great :-)
--Massimo.
Il giorno 25/ott/2011, alle ore 15.33, Olivier Delalleau ha scritto:
> Maybe try genfromtxt instead of loadtxt, it has a skip_footer option.
>
> -=- Olivier
>
> 2011/10/25 Massimo Di Stefano
> i'm tring to generate an array reading a txt file from internet.
> my target is to use python instead of matlab, to replace this steps in matlab
> :
>
> url=['http://www.cdc.noaa.gov/Correlation/amon.us.long.data'];
> urlwrite(url,'file.txt');
>
> i'm using this code :
>
> urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data',
> 'file.txt') a = np.loadtxt('file.txt', skiprows=1)
>
> but it fails becouse of the txt description at the end of the file,
>
> do you know if exist a way to skip the X lines at the end,
>
> something like "skipmultiplerows='1,-4'" (to skip the first and the last 4
> rows in the file)
>
> or i have to use some sort of string manipulation (readlines?) instead ?
>
>
> Thanks!
>
> --Massimo
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
Hi, Thank you for your gracious email. On Tue, Oct 25, 2011 at 2:56 PM, Travis Oliphant wrote: > It is a shame that Nathaniel and perhaps Matthew do not feel like their > voice was heard. I wish I could have participated more fully in some of > the discussions. I don't know if I could have really helped, but I would > have liked to have tried to perhaps work alongside Mark to integrate some of > the other ideas that had been expressed during the discussion. > Unfortunately, I was traveling in NYC most of the time that Mark was > working on this project and did not get a chance to interact with him as > much as I would have liked. > My view is that we didn't get quite to where I thought we would get, nor > where I think we could be. I think Nathaniel and Matthew provided very > specific feedback that was helpful in understanding other perspectives of a > difficult problem. In particular, I really wanted bit-patterns > implemented. However, I also understand that Mark did quite a bit of work > and altered his original designs quite a bit in response to community > feedback. I wasn't a major part of the pull request discussion, nor did I > merge the changes, but I support Charles if he reviewed the code and felt > like it was the right thing to do. I likely would have done the same thing > rather than let Mark Wiebe's work languish. > Merging Mark's code does not mean there is not more work to be done, but it > is consistent with the reality that currently development on NumPy happens > when people have the time to do it. I have not seen anything to convince > me that there is not still time to make specific API changes that address > some of the concerns. > Perhaps, Nathaniel and or Matthew could summarize their concerns again and > if desired submit a pull request to revert the changes. However, there is > a definite bias against removing working code unless the arguments are very > strong and receive a lot of support from others. Honestly - I am not sure whether there is any interest now, in the arguments we made before. If there is, who is interested? I mean, past politeness. I wasn't trying to restart that discussion, because I didn't know what good it could do. At first I was hoping that we could ask whether there was a better way of dealing with disagreements like this. Later it seemed to me that the atmosphere was getting bad, and I wanted to say that because I thought it was important. > Thank you for continuing to voice your opinions even when it may feel that > the tide is against you. My view is that we only learn from people who > disagree with us. Thank you for saying that. I hope that y'all will tell me if I am making it harder for you to disagree, and I am sorry if I did so here. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi, On Tue, Oct 25, 2011 at 2:58 PM, David Cournapeau wrote: > On Tue, Oct 25, 2011 at 8:22 PM, Matthew Brett > wrote: >> Hi, >> >> On Tue, Oct 25, 2011 at 12:14 PM, Pauli Virtanen wrote: >>> 25.10.2011 20:29, Matthew Brett kirjoitti: >>> [clip] In [7]: (res-1) / 2**32 Out[7]: 8589934591.98 In [8]: np.float((res-1) / 2**32) Out[8]: 4294967296.0 >>> >>> Looks like a bug in the C library installed on the machine, then. >>> >>> It's either in wontfix territory for us, or in the "cast to doubles >>> before formatting" one. In the latter case, one would have to maintain a >>> list of broken C libraries (ugh). >> >> How about a check at import time and a warning when printing? Is that >> hard to do? > > That's fragile IMO. I think that Chuck summed it well: long double are > not portable, don't use them unless you have to or you can rely on > platform-specificities. That reminds me of the old joke about the Irishman giving directions - "If I were you, I wouldn't start from here". > I would rather spend some time on implementing/integrating portable > quad precision in software, I guess from your answer that such a warning would be complicated to implement, and if that's the case, I can imagine it would be low priority. See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
On Tue, Oct 25, 2011 at 8:22 PM, Matthew Brett wrote: > Hi, > > On Tue, Oct 25, 2011 at 12:14 PM, Pauli Virtanen wrote: >> 25.10.2011 20:29, Matthew Brett kirjoitti: >> [clip] >>> In [7]: (res-1) / 2**32 >>> Out[7]: 8589934591.98 >>> >>> In [8]: np.float((res-1) / 2**32) >>> Out[8]: 4294967296.0 >> >> Looks like a bug in the C library installed on the machine, then. >> >> It's either in wontfix territory for us, or in the "cast to doubles >> before formatting" one. In the latter case, one would have to maintain a >> list of broken C libraries (ugh). > > How about a check at import time and a warning when printing? Is that > hard to do? That's fragile IMO. I think that Chuck summed it well: long double are not portable, don't use them unless you have to or you can rely on platform-specificities. I would rather spend some time on implementing/integrating portable quad precision in software, cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
It is a shame that Nathaniel and perhaps Matthew do not feel like their voice was heard. I wish I could have participated more fully in some of the discussions. I don't know if I could have really helped, but I would have liked to have tried to perhaps work alongside Mark to integrate some of the other ideas that had been expressed during the discussion. Unfortunately, I was traveling in NYC most of the time that Mark was working on this project and did not get a chance to interact with him as much as I would have liked. My view is that we didn't get quite to where I thought we would get, nor where I think we could be. I think Nathaniel and Matthew provided very specific feedback that was helpful in understanding other perspectives of a difficult problem. In particular, I really wanted bit-patterns implemented. However, I also understand that Mark did quite a bit of work and altered his original designs quite a bit in response to community feedback. I wasn't a major part of the pull request discussion, nor did I merge the changes, but I support Charles if he reviewed the code and felt like it was the right thing to do. I likely would have done the same thing rather than let Mark Wiebe's work languish. Merging Mark's code does not mean there is not more work to be done, but it is consistent with the reality that currently development on NumPy happens when people have the time to do it.I have not seen anything to convince me that there is not still time to make specific API changes that address some of the concerns. Perhaps, Nathaniel and or Matthew could summarize their concerns again and if desired submit a pull request to revert the changes. However, there is a definite bias against removing working code unless the arguments are very strong and receive a lot of support from others. Thank you for continuing to voice your opinions even when it may feel that the tide is against you. My view is that we only learn from people who disagree with us. Best regards, -Travis On Oct 25, 2011, at 1:24 PM, Benjamin Root wrote: > On Tue, Oct 25, 2011 at 1:03 PM, Matthew Brett > wrote: > Hi, > > On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote: > > Matthew Brett writes: > >> I'm afraid I find this whole thread very unpleasant. > > > >> I have the odd impression of being back at high school. Some of the > >> big kids are pushing me around and then the other kids join in. > > > >> It didn't have to be this way. > > > >> Someone could have replied like this to Nathaniel: > > > >> "Oh - yes - I'm sorry - we actually had the discussion on the pull > >> request. Looking back, I see that we didn't flag this up on the > >> mailing list and maybe we should have. Thanks for pointing that out. > >> Maybe we could start another discussion of the API in view of the > >> changes that have gone in". > > > >> But that didn't happen. > > > > Well, I really thought that all the interested parties would take a look at > > [1]. > > > > While it's true that the pull requests are not obvious if you're not using > > the > > functionalities of the github web (or unless announced in this list), I > > think > > that Mark's announcement was precisely directed at having a new round of > > discussions after having some code to play around with and see how > > intuitive or > > counter-intuitive the implemented concepts could be. > > I just wanted to be clear what I meant. > > The key point is not whether or not the pull-request or request for > testing was in fact the right place for the discussion that Travis > suggested. I guess you can argue that either way. I'd say no, but > I can see how you would disagree on that. > > > This is getting very meta... a disagreement about the disagreement. > > The key point is - how much do we value constructive disagreement? > > > Personally, I value it very much. My impression of the discussion we all had > at the beginning was that the needs of the two distinct communities (R-users > and masked array users) were both heard and largely addressed. Aspects of > both approaches were used, and the final result is, IMHO, inspired and > elegant. Is it perfect? No. Are there ways to improve it? Absolutely, and I > fully expect that to happen. > > If we do value constructive disagreement then we'll go out of our way > to talk through the points of contention, and make sure that the > people who disagree, especially the minority, feel that they have been > fully heard. > > If we don't value constructive disagreement then we'll let the other > side know that further disagreement will be taken as a sign of bad > faith. > > Now - what do you see here? I see the second and that worries me. > > > It is disappointing that you choose not to participate in the thread linked > above or in the pull request itself. If I remember correctly, you were > working on finishing up your dissertation, so I fully understand the time >
Re: [Numpy-discussion] NA masks in the next numpy release?
Matthew Brett writes: [...] >>> If we do value constructive disagreement then we'll go out of our way >>> to talk through the points of contention, and make sure that the >>> people who disagree, especially the minority, feel that they have been >>> fully heard. >>> >>> If we don't value constructive disagreement then we'll let the other >>> side know that further disagreement will be taken as a sign of bad >>> faith. >>> >>> Now - what do you see here? I see the second and that worries me. >>> >> >> It is disappointing that you choose not to participate in the thread linked >> above or in the pull request itself. If I remember correctly, you were >> working on finishing up your dissertation, so I fully understand the time >> constraints involved there. However, the pull request and the email >> notification is the de facto method of staging and discussing changes in any >> development project. No objections were raised in that pull request, so it >> went in after some time passed. To hold off the merge, all one would need >> to do is fire off a quick comment requesting a delay to have a chance to >> review the pull request. > I think the pull-request was not the right vehicle for the discussion, > you think it was, that's fine, I don't think we need to rehearse that. > My question (if you are answering my question) is: if you put yourself > in my or Nathaniel's shoes, would you feel that you had been warmly > encouraged to express disagreement, or would you feel something else. I sense (bear with me, my senses are not very sharp) that you feel your concerns have not been addressed, and thus the sensation that features you disagreed upon were sneaked through a silent pull request. And yes, the initial discussions were too heated on some moments (me included), but that does not imply that the current state is ignoring the concerns everybody raised. >> Luckily, git is a VCS, so we are fully capable of reverting any necessary >> changes if warranted. If you have any concerns or suggestions for changes >> in the current implementation, feel free to raise them and open additional >> pull requests. There is no "ganging up" here or any other subterfuge. Tell >> us exactly what are your issues with the current setup, provide example code >> demonstrating the issues, and we can certainly discuss ways to improve this. > Has the situation changed since the counter-NEP that Nathaniel and I wrote up? I couldn't find the link, but AFAIR the main concerns were: - Using bit patterns as a more efficient missing data mechanism that is compatible with third-party binary libraries. As the NEP says, although not implemented (due to lack of time), bit patterns are a desirable extension that will be able to coexist with masks while providing a single and consistent Python and C API for both bit patterns and masks. - Being able to expose the non-destructive nature of masks. There is only one very specific path leading to such behaviour [1], so users not interested in it should never inadvertently fall into its use (aka, they don't even need to know about it). [1] http://docs.scipy.org/doc/numpy/reference/arrays.maskna.html#creating-na-masked-views If we agree that it is reasonable to think that the concerns in the "counter-NEP" have been addressed in the current implementation, then I think it is not unreasonable to take the silence to Mark's mail and the pull request as a green light. Lluis -- "And it's much the same thing with knowledge, for whenever you learn something new, the whole world becomes that much richer." -- The Princess of Pure Reason, as told by Norton Juster in The Phantom Tollbooth ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] skip lines at the end of file with loadtxt
Maybe try genfromtxt instead of loadtxt, it has a skip_footer option.
-=- Olivier
2011/10/25 Massimo Di Stefano
> i'm tring to generate an array reading a txt file from internet.
>
> my target is to use python instead of matlab, to replace this steps in
> matlab :
>
> url=['http://www.cdc.noaa.gov/Correlation/amon.us.long.data'];
> urlwrite(url,'file.txt');
>
> i'm using this code :
>
> urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data',
> 'file.txt') a = np.loadtxt('file.txt', skiprows=1)
>
> but it fails becouse of the txt description at the end of the file,
>
> do you know if exist a way to skip the X lines at the end,
>
> something like "skipmultiplerows='1,-4'" (to skip the first and the last 4
> rows in the file)
>
> or i have to use some sort of string manipulation (readlines?) instead ?
>
> Thanks!
>
> --Massimo
>
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] skip lines at the end of file with loadtxt
i'm tring to generate an array reading a txt file from internet.
my target is to use python instead of matlab, to replace this steps in matlab :
url=['http://www.cdc.noaa.gov/Correlation/amon.us.long.data'];
urlwrite(url,'file.txt');
i'm using this code :
urllib.urlretrieve('http://www.cdc.noaa.gov/Correlation/amon.us.long.data',
'file.txt') a = np.loadtxt('file.txt', skiprows=1)
but it fails becouse of the txt description at the end of the file,
do you know if exist a way to skip the X lines at the end,
something like "skipmultiplerows='1,-4'" (to skip the first and the last 4 rows
in the file)
or i have to use some sort of string manipulation (readlines?) instead ?
Thanks!
--Massimo___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi, On Tue, Oct 25, 2011 at 12:14 PM, Pauli Virtanen wrote: > 25.10.2011 20:29, Matthew Brett kirjoitti: > [clip] >> In [7]: (res-1) / 2**32 >> Out[7]: 8589934591.98 >> >> In [8]: np.float((res-1) / 2**32) >> Out[8]: 4294967296.0 > > Looks like a bug in the C library installed on the machine, then. > > It's either in wontfix territory for us, or in the "cast to doubles > before formatting" one. In the latter case, one would have to maintain a > list of broken C libraries (ugh). How about a check at import time and a warning when printing? Is that hard to do? See you, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi, On 25 Oct 2011, at 21:14, Pauli Virtanen wrote: > 25.10.2011 20:29, Matthew Brett kirjoitti: > [clip] >> In [7]: (res-1) / 2**32 >> Out[7]: 8589934591.98 >> >> In [8]: np.float((res-1) / 2**32) >> Out[8]: 4294967296.0 > > Looks like a bug in the C library installed on the machine, then. > > It's either in wontfix territory for us, or in the "cast to doubles > before formatting" one. In the latter case, one would have to maintain a > list of broken C libraries (ugh). as it appears to be a "Tiger-only" problem, probably the former? On 25 Oct 2011, at 21:13, Matthew Brett wrote: > [mb312@jerry ~]$ uname -a > Darwin jerry.bic.berkeley.edu 8.11.0 Darwin Kernel Version 8.11.0: Wed > Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power > Macintosh powerpc Cheers, Derek ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
25.10.2011 20:29, Matthew Brett kirjoitti: [clip] > In [7]: (res-1) / 2**32 > Out[7]: 8589934591.98 > > In [8]: np.float((res-1) / 2**32) > Out[8]: 4294967296.0 Looks like a bug in the C library installed on the machine, then. It's either in wontfix territory for us, or in the "cast to doubles before formatting" one. In the latter case, one would have to maintain a list of broken C libraries (ugh). -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi,
On Tue, Oct 25, 2011 at 12:01 PM, Derek Homeier
wrote:
> On 25 Oct 2011, at 20:05, Matthew Brett wrote:
>
Both the same as numpy:
[mb312@jerry ~]$ gcc test.c
test.c: In function 'main':
test.c:5: warning: incompatible implicit declaration of built-in function
'powl'
>>>
>>> I think implicit here means that that the arguments and the return values
>>> are treated as integers. Did you #include ?
>>
>> Ah - you've detected my severe ignorance of c. But with math.h, the
>> result is the same,
>>
>> #include
>> #include
>>
>> int main(int argc, char* argv) {
>> long double x;
>> x = pow(2, 64);
>> x -= 1;
>> printf("%g %Lg\n", (double)x, x);
>> }
>
> What system/compiler is this? I am getting
> ./ldouble
> 1.84467e+19 1.84467e+19
>
> and
>
res = np.longdouble(2)**64
res
> 18446744073709551616.0
2**64
> 18446744073709551616L
res-1
> 18446744073709551615.0
np.__version__
> '1.6.1'
>
> as well as with
>
np.__version__
> '2.0.0.dev-3d06f02'
> [yes, not very up to date]
>
> and for all gcc versions
> /usr/bin/gcc -v
> Using built-in specs.
> Target: powerpc-apple-darwin9
> Configured with: /var/tmp/gcc/gcc-5493~1/src/configure --disable-checking
> -enable-werror --prefix=/usr --mandir=/share/man
> --enable-languages=c,objc,c++,obj-c++
> --program-transform-name=/^[cg][^.-]*$/s/$/-4.0/
> --with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib
> --build=i686-apple-darwin9 --program-prefix= --host=powerpc-apple-darwin9
> --target=powerpc-apple-darwin9
> Thread model: posix
> gcc version 4.0.1 (Apple Inc. build 5493)
>
> to
>
> /sw/bin/gcc-fsf-4.6 -v
> Using built-in specs.
> COLLECT_GCC=/sw/bin/gcc-fsf-4.6
> COLLECT_LTO_WRAPPER=/sw/lib/gcc4.6/libexec/gcc/powerpc-apple-darwin9.8.0/4.6.1/lto-wrapper
> Target: powerpc-apple-darwin9.8.0
> Configured with: ../gcc-4.6.1/configure --prefix=/sw --prefix=/sw/lib/gcc4.6
> --mandir=/sw/share/man --infodir=/sw/lib/gcc4.6/info
> --enable-languages=c,c++,fortran,lto,objc,obj-c++,java --with-gmp=/sw
> --with-libiconv-prefix=/sw --with-ppl=/sw --with-cloog=/sw --with-mpc=/sw
> --with-system-zlib --x-includes=/usr/X11R6/include
> --x-libraries=/usr/X11R6/lib --program-suffix=-fsf-4.6
> --enable-cloog-backend=isl --disable-libjava-multilib --disable-libquadmath
> Thread model: posix
> gcc version 4.6.1 (GCC)
>
> uname -a
> Darwin osiris.astro.physik.uni-goettingen.de 9.8.0 Darwin Kernel Version
> 9.8.0: Wed Jul 15 16:57:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_PPC Power
> Macintosh
mb312@jerry ~]$ gcc -v
Using built-in specs.
Target: powerpc-apple-darwin8
Configured with: /var/tmp/gcc/gcc-5370~2/src/configure
--disable-checking -enable-werror --prefix=/usr --mandir=/share/man
--enable-languages=c,objc,c++,obj-c++
--program-transform-name=/^[cg][^.-]*$/s/$/-4.0/
--with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib
--build=powerpc-apple-darwin8 --host=powerpc-apple-darwin8
--target=powerpc-apple-darwin8
Thread model: posix
gcc version 4.0.1 (Apple Computer, Inc. build 5370)
[mb312@jerry ~]$ uname -a
Darwin jerry.bic.berkeley.edu 8.11.0 Darwin Kernel Version 8.11.0: Wed
Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power
Macintosh powerpc
Best,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
Hi, On Tue, Oct 25, 2011 at 11:24 AM, Benjamin Root wrote: > On Tue, Oct 25, 2011 at 1:03 PM, Matthew Brett > wrote: >> >> Hi, >> >> On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote: >> > Matthew Brett writes: >> >> I'm afraid I find this whole thread very unpleasant. >> > >> >> I have the odd impression of being back at high school. Some of the >> >> big kids are pushing me around and then the other kids join in. >> > >> >> It didn't have to be this way. >> > >> >> Someone could have replied like this to Nathaniel: >> > >> >> "Oh - yes - I'm sorry - we actually had the discussion on the pull >> >> request. Looking back, I see that we didn't flag this up on the >> >> mailing list and maybe we should have. Thanks for pointing that out. >> >> Maybe we could start another discussion of the API in view of the >> >> changes that have gone in". >> > >> >> But that didn't happen. >> > >> > Well, I really thought that all the interested parties would take a look >> > at [1]. >> > >> > While it's true that the pull requests are not obvious if you're not >> > using the >> > functionalities of the github web (or unless announced in this list), I >> > think >> > that Mark's announcement was precisely directed at having a new round of >> > discussions after having some code to play around with and see how >> > intuitive or >> > counter-intuitive the implemented concepts could be. >> >> I just wanted to be clear what I meant. >> >> The key point is not whether or not the pull-request or request for >> testing was in fact the right place for the discussion that Travis >> suggested. I guess you can argue that either way. I'd say no, but >> I can see how you would disagree on that. >> > > This is getting very meta... a disagreement about the disagreement. Yes, the important point is a social one. The other points are details. >> The key point is - how much do we value constructive disagreement? >> > > Personally, I value it very much. Well - I think everyone believes that that they value constructive discussion, but the question is, what happens when people really disagree? > My impression of the discussion we all > had at the beginning was that the needs of the two distinct communities > (R-users and masked array users) were both heard and largely addressed. > Aspects of both approaches were used, and the final result is, IMHO, > inspired and elegant. Is it perfect? No. Are there ways to improve it? > Absolutely, and I fully expect that to happen. To be clear once more, I personally feel we don't need to discuss: 1) Whether Mark did a good job on the code (I have high bias to imagine so). 2) Whether something along these lines would be good to have in numpy >> If we do value constructive disagreement then we'll go out of our way >> to talk through the points of contention, and make sure that the >> people who disagree, especially the minority, feel that they have been >> fully heard. >> >> If we don't value constructive disagreement then we'll let the other >> side know that further disagreement will be taken as a sign of bad >> faith. >> >> Now - what do you see here? I see the second and that worries me. >> > > It is disappointing that you choose not to participate in the thread linked > above or in the pull request itself. If I remember correctly, you were > working on finishing up your dissertation, so I fully understand the time > constraints involved there. However, the pull request and the email > notification is the de facto method of staging and discussing changes in any > development project. No objections were raised in that pull request, so it > went in after some time passed. To hold off the merge, all one would need > to do is fire off a quick comment requesting a delay to have a chance to > review the pull request. I think the pull-request was not the right vehicle for the discussion, you think it was, that's fine, I don't think we need to rehearse that. My question (if you are answering my question) is: if you put yourself in my or Nathaniel's shoes, would you feel that you had been warmly encouraged to express disagreement, or would you feel something else. > Luckily, git is a VCS, so we are fully capable of reverting any necessary > changes if warranted. If you have any concerns or suggestions for changes > in the current implementation, feel free to raise them and open additional > pull requests. There is no "ganging up" here or any other subterfuge. Tell > us exactly what are your issues with the current setup, provide example code > demonstrating the issues, and we can certainly discuss ways to improve this. Has the situation changed since the counter-NEP that Nathaniel and I wrote up? > Remember, we *all* have a common agreement here. NumPy needs better support > for missing data (in whatever form). Let's work from that assumption and > make NumPy a better library to use for everybody! I remember walking past a church in a small town in the California desert. It ha
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
On 25 Oct 2011, at 20:05, Matthew Brett wrote:
>>> Both the same as numpy:
>>>
>>> [mb312@jerry ~]$ gcc test.c
>>> test.c: In function 'main':
>>> test.c:5: warning: incompatible implicit declaration of built-in function
>>> 'powl'
>>
>> I think implicit here means that that the arguments and the return values
>> are treated as integers. Did you #include ?
>
> Ah - you've detected my severe ignorance of c. But with math.h, the
> result is the same,
>
> #include
> #include
>
> int main(int argc, char* argv) {
> long double x;
> x = pow(2, 64);
> x -= 1;
> printf("%g %Lg\n", (double)x, x);
> }
What system/compiler is this? I am getting
./ldouble
1.84467e+19 1.84467e+19
and
>>> res = np.longdouble(2)**64
>>> res
18446744073709551616.0
>>> 2**64
18446744073709551616L
>>> res-1
18446744073709551615.0
>>> np.__version__
'1.6.1'
as well as with
>>> np.__version__
'2.0.0.dev-3d06f02'
[yes, not very up to date]
and for all gcc versions
/usr/bin/gcc -v
Using built-in specs.
Target: powerpc-apple-darwin9
Configured with: /var/tmp/gcc/gcc-5493~1/src/configure --disable-checking
-enable-werror --prefix=/usr --mandir=/share/man
--enable-languages=c,objc,c++,obj-c++
--program-transform-name=/^[cg][^.-]*$/s/$/-4.0/
--with-gxx-include-dir=/include/c++/4.0.0 --with-slibdir=/usr/lib
--build=i686-apple-darwin9 --program-prefix= --host=powerpc-apple-darwin9
--target=powerpc-apple-darwin9
Thread model: posix
gcc version 4.0.1 (Apple Inc. build 5493)
to
/sw/bin/gcc-fsf-4.6 -v
Using built-in specs.
COLLECT_GCC=/sw/bin/gcc-fsf-4.6
COLLECT_LTO_WRAPPER=/sw/lib/gcc4.6/libexec/gcc/powerpc-apple-darwin9.8.0/4.6.1/lto-wrapper
Target: powerpc-apple-darwin9.8.0
Configured with: ../gcc-4.6.1/configure --prefix=/sw --prefix=/sw/lib/gcc4.6
--mandir=/sw/share/man --infodir=/sw/lib/gcc4.6/info
--enable-languages=c,c++,fortran,lto,objc,obj-c++,java --with-gmp=/sw
--with-libiconv-prefix=/sw --with-ppl=/sw --with-cloog=/sw --with-mpc=/sw
--with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib
--program-suffix=-fsf-4.6 --enable-cloog-backend=isl --disable-libjava-multilib
--disable-libquadmath
Thread model: posix
gcc version 4.6.1 (GCC)
uname -a
Darwin osiris.astro.physik.uni-goettingen.de 9.8.0 Darwin Kernel Version 9.8.0:
Wed Jul 15 16:57:01 PDT 2009; root:xnu-1228.15.4~1/RELEASE_PPC Power Macintosh
Cheers,
Derek
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi,
On Tue, Oct 25, 2011 at 11:14 AM, Pauli Virtanen wrote:
> 25.10.2011 19:45, Matthew Brett kirjoitti:
> [clip]
>>> or, in case the platform doesn't have powl:
>>>
>>> long double x;
>>> x = pow(2, 64);
>>> x -= 1;
>>> printf("%g %Lg\n", (double)x, x);
>>
>> Both the same as numpy:
>>
>> [mb312@jerry ~]$ gcc test.c
>> test.c: In function 'main':
>> test.c:5: warning: incompatible implicit declaration of built-in function
>> 'powl'
>> [mb312@jerry ~]$ ./a.out
>> 1.84467e+19 3.68935e+19
>
> This result may indicate that it's the *printing* of long doubles that's
> broken. Note how the value cast as double prints the correct result,
> whereas the %Lg format code gives something wrong.
Ah - sorry - I see now what you were trying to do.
> Can you try to check this by doing something like:
>
> - do some set of calculations using np.longdouble in Numpy
> (that requires the extra accuracy)
>
> - at the end, cast the result back to double
In [1]: import numpy as np
In [2]: res = np.longdouble(2)**64
In [6]: res / 2**32
Out[6]: 4294967296.0
In [7]: (res-1) / 2**32
Out[7]: 8589934591.98
In [8]: np.float((res-1) / 2**32)
Out[8]: 4294967296.0
In [9]: np.float((res) / 2**32)
Out[9]: 4294967296.0
Thanks,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
On Tue, Oct 25, 2011 at 1:03 PM, Matthew Brett wrote: > Hi, > > On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote: > > Matthew Brett writes: > >> I'm afraid I find this whole thread very unpleasant. > > > >> I have the odd impression of being back at high school. Some of the > >> big kids are pushing me around and then the other kids join in. > > > >> It didn't have to be this way. > > > >> Someone could have replied like this to Nathaniel: > > > >> "Oh - yes - I'm sorry - we actually had the discussion on the pull > >> request. Looking back, I see that we didn't flag this up on the > >> mailing list and maybe we should have. Thanks for pointing that out. > >> Maybe we could start another discussion of the API in view of the > >> changes that have gone in". > > > >> But that didn't happen. > > > > Well, I really thought that all the interested parties would take a look > at [1]. > > > > While it's true that the pull requests are not obvious if you're not > using the > > functionalities of the github web (or unless announced in this list), I > think > > that Mark's announcement was precisely directed at having a new round of > > discussions after having some code to play around with and see how > intuitive or > > counter-intuitive the implemented concepts could be. > > I just wanted to be clear what I meant. > > The key point is not whether or not the pull-request or request for > testing was in fact the right place for the discussion that Travis > suggested. I guess you can argue that either way. I'd say no, but > I can see how you would disagree on that. > > This is getting very meta... a disagreement about the disagreement. > The key point is - how much do we value constructive disagreement? > > Personally, I value it very much. My impression of the discussion we all had at the beginning was that the needs of the two distinct communities (R-users and masked array users) were both heard and largely addressed. Aspects of both approaches were used, and the final result is, IMHO, inspired and elegant. Is it perfect? No. Are there ways to improve it? Absolutely, and I fully expect that to happen. > If we do value constructive disagreement then we'll go out of our way > to talk through the points of contention, and make sure that the > people who disagree, especially the minority, feel that they have been > fully heard. > > If we don't value constructive disagreement then we'll let the other > side know that further disagreement will be taken as a sign of bad > faith. > > Now - what do you see here? I see the second and that worries me. > > It is disappointing that you choose not to participate in the thread linked above or in the pull request itself. If I remember correctly, you were working on finishing up your dissertation, so I fully understand the time constraints involved there. However, the pull request and the email notification is the de facto method of staging and discussing changes in any development project. No objections were raised in that pull request, so it went in after some time passed. To hold off the merge, all one would need to do is fire off a quick comment requesting a delay to have a chance to review the pull request. Luckily, git is a VCS, so we are fully capable of reverting any necessary changes if warranted. If you have any concerns or suggestions for changes in the current implementation, feel free to raise them and open additional pull requests. There is no "ganging up" here or any other subterfuge. Tell us exactly what are your issues with the current setup, provide example code demonstrating the issues, and we can certainly discuss ways to improve this. Remember, we *all* have a common agreement here. NumPy needs better support for missing data (in whatever form). Let's work from that assumption and make NumPy a better library to use for everybody! Cheers! Ben Root ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
25.10.2011 19:45, Matthew Brett kirjoitti:
[clip]
>> or, in case the platform doesn't have powl:
>>
>> long double x;
>> x = pow(2, 64);
>> x -= 1;
>> printf("%g %Lg\n", (double)x, x);
>
> Both the same as numpy:
>
> [mb312@jerry ~]$ gcc test.c
> test.c: In function 'main':
> test.c:5: warning: incompatible implicit declaration of built-in function
> 'powl'
> [mb312@jerry ~]$ ./a.out
> 1.84467e+19 3.68935e+19
This result may indicate that it's the *printing* of long doubles that's
broken. Note how the value cast as double prints the correct result,
whereas the %Lg format code gives something wrong.
Can you try to check this by doing something like:
- do some set of calculations using np.longdouble in Numpy
(that requires the extra accuracy)
- at the end, cast the result back to double
--
Pauli Virtanen
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
On Tue, Oct 25, 2011 at 11:05 AM, Matthew Brett wrote:
> Hi,
>
> On Tue, Oct 25, 2011 at 10:52 AM, Charles R Harris
> wrote:
>>
>>
>> On Tue, Oct 25, 2011 at 11:45 AM, Matthew Brett
>> wrote:
>>>
>>> Hi,
>>>
>>> On Tue, Oct 25, 2011 at 2:43 AM, Pauli Virtanen wrote:
>>> > 25.10.2011 06:59, Matthew Brett kirjoitti:
>>> >> res = np.longdouble(2)**64
>>> >> res-1
>>> >> 36893488147419103231.0
>>> >
>>> > Can you check if long double works properly (not a given) in C on that
>>> > platform:
>>> >
>>> > long double x;
>>> > x = powl(2, 64);
>>> > x -= 1;
>>> > printf("%g %Lg\n", (double)x, x);
>>> >
>>> > or, in case the platform doesn't have powl:
>>> >
>>> > long double x;
>>> > x = pow(2, 64);
>>> > x -= 1;
>>> > printf("%g %Lg\n", (double)x, x);
>>>
>>> Both the same as numpy:
>>>
>>> [mb312@jerry ~]$ gcc test.c
>>> test.c: In function 'main':
>>> test.c:5: warning: incompatible implicit declaration of built-in function
>>> 'powl'
>>
>> I think implicit here means that that the arguments and the return values
>> are treated as integers. Did you #include ?
>
> Ah - you've detected my severe ignorance of c. But with math.h, the
> result is the same,
>
> #include
> #include
>
> int main(int argc, char* argv) {
> long double x;
> x = pow(2, 64);
> x -= 1;
> printf("%g %Lg\n", (double)x, x);
> }
By the way - if you want a login to this machine, let me know - it's
always on and we're using it as a buildslave already.
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi,
On Tue, Oct 25, 2011 at 10:52 AM, Charles R Harris
wrote:
>
>
> On Tue, Oct 25, 2011 at 11:45 AM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> On Tue, Oct 25, 2011 at 2:43 AM, Pauli Virtanen wrote:
>> > 25.10.2011 06:59, Matthew Brett kirjoitti:
>> >> res = np.longdouble(2)**64
>> >> res-1
>> >> 36893488147419103231.0
>> >
>> > Can you check if long double works properly (not a given) in C on that
>> > platform:
>> >
>> > long double x;
>> > x = powl(2, 64);
>> > x -= 1;
>> > printf("%g %Lg\n", (double)x, x);
>> >
>> > or, in case the platform doesn't have powl:
>> >
>> > long double x;
>> > x = pow(2, 64);
>> > x -= 1;
>> > printf("%g %Lg\n", (double)x, x);
>>
>> Both the same as numpy:
>>
>> [mb312@jerry ~]$ gcc test.c
>> test.c: In function 'main':
>> test.c:5: warning: incompatible implicit declaration of built-in function
>> 'powl'
>
> I think implicit here means that that the arguments and the return values
> are treated as integers. Did you #include ?
Ah - you've detected my severe ignorance of c. But with math.h, the
result is the same,
#include
#include
int main(int argc, char* argv) {
long double x;
x = pow(2, 64);
x -= 1;
printf("%g %Lg\n", (double)x, x);
}
See you,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
Hi, On Tue, Oct 25, 2011 at 8:04 AM, Lluís wrote: > Matthew Brett writes: >> I'm afraid I find this whole thread very unpleasant. > >> I have the odd impression of being back at high school. Some of the >> big kids are pushing me around and then the other kids join in. > >> It didn't have to be this way. > >> Someone could have replied like this to Nathaniel: > >> "Oh - yes - I'm sorry - we actually had the discussion on the pull >> request. Looking back, I see that we didn't flag this up on the >> mailing list and maybe we should have. Thanks for pointing that out. >> Maybe we could start another discussion of the API in view of the >> changes that have gone in". > >> But that didn't happen. > > Well, I really thought that all the interested parties would take a look at > [1]. > > While it's true that the pull requests are not obvious if you're not using the > functionalities of the github web (or unless announced in this list), I think > that Mark's announcement was precisely directed at having a new round of > discussions after having some code to play around with and see how intuitive > or > counter-intuitive the implemented concepts could be. I just wanted to be clear what I meant. The key point is not whether or not the pull-request or request for testing was in fact the right place for the discussion that Travis suggested. I guess you can argue that either way. I'd say no, but I can see how you would disagree on that. The key point is - how much do we value constructive disagreement? If we do value constructive disagreement then we'll go out of our way to talk through the points of contention, and make sure that the people who disagree, especially the minority, feel that they have been fully heard. If we don't value constructive disagreement then we'll let the other side know that further disagreement will be taken as a sign of bad faith. Now - what do you see here? I see the second and that worries me. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
On Tue, Oct 25, 2011 at 11:45 AM, Matthew Brett wrote:
> Hi,
>
> On Tue, Oct 25, 2011 at 2:43 AM, Pauli Virtanen wrote:
> > 25.10.2011 06:59, Matthew Brett kirjoitti:
> >> res = np.longdouble(2)**64
> >> res-1
> >> 36893488147419103231.0
> >
> > Can you check if long double works properly (not a given) in C on that
> > platform:
> >
> >long double x;
> >x = powl(2, 64);
> >x -= 1;
> >printf("%g %Lg\n", (double)x, x);
> >
> > or, in case the platform doesn't have powl:
> >
> >long double x;
> >x = pow(2, 64);
> >x -= 1;
> >printf("%g %Lg\n", (double)x, x);
>
> Both the same as numpy:
>
> [mb312@jerry ~]$ gcc test.c
> test.c: In function 'main':
> test.c:5: warning: incompatible implicit declaration of built-in function
> 'powl'
>
I think implicit here means that that the arguments and the return values
are treated as integers. Did you #include ?
> [mb312@jerry ~]$ ./a.out
> 1.84467e+19 3.68935e+19
>
>
Chuck
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi,
On Tue, Oct 25, 2011 at 2:43 AM, Pauli Virtanen wrote:
> 25.10.2011 06:59, Matthew Brett kirjoitti:
>> res = np.longdouble(2)**64
>> res-1
>> 36893488147419103231.0
>
> Can you check if long double works properly (not a given) in C on that
> platform:
>
> long double x;
> x = powl(2, 64);
> x -= 1;
> printf("%g %Lg\n", (double)x, x);
>
> or, in case the platform doesn't have powl:
>
> long double x;
> x = pow(2, 64);
> x -= 1;
> printf("%g %Lg\n", (double)x, x);
Both the same as numpy:
[mb312@jerry ~]$ gcc test.c
test.c: In function 'main':
test.c:5: warning: incompatible implicit declaration of built-in function 'powl'
[mb312@jerry ~]$ ./a.out
1.84467e+19 3.68935e+19
Thanks,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
Hi, On Tue, Oct 25, 2011 at 7:31 AM, Charles R Harris wrote: > > > On Mon, Oct 24, 2011 at 10:59 PM, Matthew Brett > wrote: >> >> Hi, >> >> I just ran into this on a PPC machine: >> >> In [1]: import numpy as np >> >> In [2]: np.__version__ >> Out[2]: '2.0.0.dev-4daf949' >> >> In [3]: res = np.longdouble(2)**64 >> >> In [4]: res >> Out[4]: 18446744073709551616.0 >> >> In [5]: 2**64 >> Out[5]: 18446744073709551616L >> >> In [6]: res-1 >> Out[6]: 36893488147419103231.0 >> >> Same for numpy 1.4.1. >> >> I don't have a SPARC to test on but I believe it's the same double-double >> type? >> > > The PPC uses two doubles to represent long doubles, the SPARC uses software > emulation of ieee quad precision for long doubles, very different. Yes, thanks - I read more after my post. I guess from this: http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.genprogc/doc/genprogc/128bit_long_double_floating-point_datatype.htm that AIX does use double-double. > The > subtraction of 1 working like multiplication by two is strange, perhaps the > one is getting subtracted from the exponent somehow? It would be interesting > to see if the same problem happens in pure c. > > As a work around, can I ask what you are trying to do with the long doubles? I was trying to use them as an intermediate format for high-precision floating point calculations, before converting to integers. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy.matrix subclassing
On Mon, Oct 24, 2011 at 5:54 PM, David Voong wrote: > Hi guys, > > I have a question regarding subclassing of the numpy.matrix class. > > I read through the wiki page, > http://docs.scipy.org/doc/numpy/user/basics.subclassing.html > > and tried to subclass numpy.matrix, I find that if I override the > __finalize_array__ method I have problems using the sum method and get the > following error, > > > Traceback (most recent call last): > File "test.py", line 61, in > print (a * b).sum() > File "/afs/ > cern.ch/user/d/dvoong/programs/lib/python2.6/site-packages/numpy/matrixlib/defmatrix.py", > line 435, in sum > return N.ndarray.sum(self, axis, dtype, out)._align(axis) > File "/afs/ > cern.ch/user/d/dvoong/programs/lib/python2.6/site-packages/numpy/matrixlib/defmatrix.py", > line 370, in _align > return self[0,0] > File "/afs/ > cern.ch/user/d/dvoong/programs/lib/python2.6/site-packages/numpy/matrixlib/defmatrix.py", > line 305, in __getitem__ > out = N.ndarray.__getitem__(self, index) > IndexError: 0-d arrays can only use a single () or a list of newaxes (and a > single ...) as an index > > > Hi, Thanks for asking this - I'm also trying to subclass np.matrix and running into similar problems; I never generally need to sum my vectors so this wasn't a problem I had noticed thus far. Anyway, for np.matrix, there are definitely particular issues beyond what is described on the array subclassing wiki. I think I have a workaround, based on struggling with my own subclass. This is really a hack since I'm not sure how some parts of matrix actually work, so if someone has a better solution please speak up! You didn't give details on the actual subclass, but I can recreate the error with the following minimal example (testing with Numpy 1.6.1 inside EPD 7.1): class MatSubClass1(np.matrix): def __new__(cls, input_array): obj = np.asarray(input_array).view(cls) return obj def __array_finalize__(self, obj): pass def __array_wrap__(self, out_arr, context=None): return np.ndarray.__array_wrap__(self, out_arr, context) In [2]: m1 = MatSubClass1( [[2,0],[1,1]] ) In [3]: m1.sum() ... IndexError: 0-d arrays can only use a single () or a list of newaxes (and a single ...) as an index The problem is that __array_finalize__ of the matrix class that needs to get called, to preserve dimensions (matrix should always have 2 dimensions). You can't just add the matrix __array_finalize__ because the initial call happens when you create the object, in which case obj is a ndarray object, not a matrix. So, check to see obj is a matrix first before calling it. In addition, there is some undocumented _getitem attribute inside matrix, and I do not know what it does. If you just set that attribute during __new__, you get something that seems to work: class MatSubClass2(np.matrix): def __new__(cls, input_array): obj = np.asarray(input_array).view(cls) obj._getitem = False return obj def __array_finalize__(self, obj): if isinstance(obj, np.matrix): np.matrix.__array_finalize__(self, obj) def __array_wrap__(self, out_arr, context=None): return np.ndarray.__array_wrap__(self, out_arr, context) In [4]: m2 = MatSubClass2( [[2,0],[1,1]] ) In [5]: m2.sum(), m2.sum(0), m2.sum(1) Out[5]: (4, matrix([[3, 1]]), matrix([[2], [2]])) HTH, Aronne ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
Matthew Brett writes: > I'm afraid I find this whole thread very unpleasant. > I have the odd impression of being back at high school. Some of the > big kids are pushing me around and then the other kids join in. > It didn't have to be this way. > Someone could have replied like this to Nathaniel: > "Oh - yes - I'm sorry - we actually had the discussion on the pull > request. Looking back, I see that we didn't flag this up on the > mailing list and maybe we should have. Thanks for pointing that out. > Maybe we could start another discussion of the API in view of the > changes that have gone in". > But that didn't happen. Well, I really thought that all the interested parties would take a look at [1]. While it's true that the pull requests are not obvious if you're not using the functionalities of the github web (or unless announced in this list), I think that Mark's announcement was precisely directed at having a new round of discussions after having some code to play around with and see how intuitive or counter-intuitive the implemented concepts could be. [1] http://old.nabble.com/NA-masks-for-NumPy-are-ready-to-test-td32291024.html Lluis -- "And it's much the same thing with knowledge, for whenever you learn something new, the whole world becomes that much richer." -- The Princess of Pure Reason, as told by Norton Juster in The Phantom Tollbooth ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
On Mon, Oct 24, 2011 at 10:59 PM, Matthew Brett wrote: > Hi, > > I just ran into this on a PPC machine: > > In [1]: import numpy as np > > In [2]: np.__version__ > Out[2]: '2.0.0.dev-4daf949' > > In [3]: res = np.longdouble(2)**64 > > In [4]: res > Out[4]: 18446744073709551616.0 > > In [5]: 2**64 > Out[5]: 18446744073709551616L > > In [6]: res-1 > Out[6]: 36893488147419103231.0 > > Same for numpy 1.4.1. > > I don't have a SPARC to test on but I believe it's the same double-double > type? > > The PPC uses two doubles to represent long doubles, the SPARC uses software emulation of ieee quad precision for long doubles, very different. The subtraction of 1 working like multiplication by two is strange, perhaps the one is getting subtracted from the exponent somehow? It would be interesting to see if the same problem happens in pure c. As a work around, can I ask what you are trying to do with the long doubles? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] neighborhood iterator speed
Finally managed to use PyArrayNeighborhoodIter_Next2D with numpy 1.5.0 (in numpy 1.6 it doesn't get along with halffloat). Benchmark results (not the same computer and parameters I used in the previous benchmark): 1. ...Next2D (zero padding, it doesn't accept mirror padding): 10 sec 2. ...Next (zero padding): 53 sec 3. ...Next (mirror padding): 128 sec Remarks: 1. I did not check the validity of the results 2. Mirror padding is preferable for my specific case. What does it mean for the potential for the neighbourhood iterator acceleration? Nadav. -Original Message- From: numpy-discussion-boun...@scipy.org [mailto:numpy-discussion-boun...@scipy.org] On Behalf Of Nadav Horesh Sent: Monday, October 24, 2011 9:02 PM To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] neighborhood iterator speed I found the 2d iterator definition active in numpy 1.6.1. I'll test it. Nadav From: numpy-discussion-boun...@scipy.org [numpy-discussion-boun...@scipy.org] On Behalf Of David Cournapeau [courn...@gmail.com] Sent: 24 October 2011 16:04 To: Discussion of Numerical Python Subject: Re: [Numpy-discussion] neighborhood iterator speed On Mon, Oct 24, 2011 at 1:23 PM, Nadav Horesh wrote: > * I'll try to implement the 2D iterator as far as far as my programming > expertise goes. It might take few days. I am pretty sure the code is in the history, if you are patient enough to look for it in git history. I can't remember why I removed it (maybe because it was not faster ?). > > * There is a risk in providing a buffer pointer, and for my (and probably > most) use cases it is better for the iterator constructor to provide it. I > was thinking about the possibility to give the iterator a shared memory > pointer, to open a door for multiprocessing. Maybe it is better instead to > provide a contiguous ndarray object to enable a sanity check. One could ask for an optional buffer (if NULL -> auto-allocation). But I would need a more detailed explanation about what you are trying to do to warrant changing the API here. cheers, David ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion __ Information from ESET NOD32 Antivirus, version of virus signature database 4628 (20091122) __ The message was checked by ESET NOD32 Antivirus. http://www.eset.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] float128 / longdouble on PPC - is it broken?
25.10.2011 06:59, Matthew Brett kirjoitti:
> res = np.longdouble(2)**64
> res-1
> 36893488147419103231.0
Can you check if long double works properly (not a given) in C on that
platform:
long double x;
x = powl(2, 64);
x -= 1;
printf("%g %Lg\n", (double)x, x);
or, in case the platform doesn't have powl:
long double x;
x = pow(2, 64);
x -= 1;
printf("%g %Lg\n", (double)x, x);
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] NA masks in the next numpy release?
Hi, On Mon, Oct 24, 2011 at 11:44 PM, Han Genuit wrote: > Well, if I may have a say, I think that an open source project is > especially open when users as developers can contribute to the code > base and can participate in discussions on how to improve the existing > designs and ideas. I do not think a project is open when it crumbles > down into politics.. I have seen a lot of work done by Mark especially > to ensure that everyone had a say in what he was doing, up to the > point where this might not be fun anymore. And from what I can see at > the time, which was back in August, everyone has had plenty of > opportunity to discuss or contribute to the specific changes that were > made. > > This was an open contribution to the NumPy code, not some cooked up > shady business by high and mighty developers and I, for one, am happy > with how it turned out. I'm afraid I find this whole thread very unpleasant. I have the odd impression of being back at high school. Some of the big kids are pushing me around and then the other kids join in. It didn't have to be this way. Someone could have replied like this to Nathaniel: "Oh - yes - I'm sorry - we actually had the discussion on the pull request. Looking back, I see that we didn't flag this up on the mailing list and maybe we should have. Thanks for pointing that out. Maybe we could start another discussion of the API in view of the changes that have gone in". But that didn't happen. Best, Matthew ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
