Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs
On 11/9/06, Robert Kern [EMAIL PROTECTED] wrote: Fernando Perez wrote: I understand why this happens, but I wonder if it should be in any way 'fixed' (if that is even feasible without introducing other problems): [...] I am sure it will be, to say the least, pretty surprising (and I can imagine subtle bugs being caused by this). I think we decided long ago that an int32 really is an array of 32-bit integers and behaves like one. That's precisely why one uses int32 arrays rather than object arrays. There are algorithms that do need the wraparound, and the Python int behavior is always available through object arrays. Mkay, fine by me. Novus caveat emptor. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Int32s, scalar operations and Pyhon longs
On 11/9/06, Tim Hochberg [EMAIL PROTECTED] wrote: Let me add that I can't imagine that the bugs will be all that subtle given that numpy now spits out a warning on overflow. If you're really worried about this I suggest you crank up the error mode to make this an error - then you really won't be able to miss this kind of overflow. Not all computations are run interactively with a human in front of the console to actually see the warnings go by. If they get buried in a log file on a cluster, there is a possibility you might miss them. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
[Numpy-discussion] [OT] Why Python 'fits your brain'
Please forgive the not-specifically-numpy post. I'll keep it short. Some of us often, when trying to explain to newcomers the benefits of Python for scientific work, use expressions like the famous 'it fits your brain'. This is an attempt at conveying why it seems like such a natural tool for expressing, concisely and without extraneous noise, but simultaneously with ease, the nature of many algorithmic problems. I think these four images http://flickr.com/photos/nicksieger/281055485/ http://flickr.com/photos/nicksieger/281055530/ http://flickr.com/photos/nicksieger/280662707/ http://flickr.com/photos/nicksieger/280661836/ may provide a useful visual aid. I certainly intend to use them (with attribution) the next time I need to give a talk along those lines. Regards, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
[Numpy-discussion] Profiling Python codes with hotshot and KCachegrind
Hi all, in the past, Arnd Baecker has made a number of very useful posts on this matter, and provided some nice utilities to do it. I now needed to profile some fairly complex codes prior to a big optimization push, so I went over his material and wrote a little tool to make the whole process as painless as possible. Here it is, hoping others may find it useful: http://amath.colorado.edu/faculty/fperez/python/profiling/ Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Profiling Python codes with hotshot and KCachegrind
On 11/8/06, Stefan van der Walt [EMAIL PROTECTED] wrote:
This looks very interesting. It works for me on simple scripts, but
whenever I include the lines
from numpy.testing import set_local_path
set_local_path('../../..')
in the input, pycachegrind aborts with
File
/home/stefan//lib/python2.4/site-packages/numpy/testing/numpytest.py, line
68, in set_local_path
if f.f_locals['__name__']=='__main__':
KeyError: '__name__'
I guess this is because the script is run in a separate namespace.
I've managed to work around the problem by changing the definition of
'run' to:
Good catch, thanks, I've fixed the public version with your changes.
Best,
f
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange and hard to reproduce crash
On 10/23/06, Travis Oliphant [EMAIL PROTECTED] wrote:
I've placed them in SVN (r3384):
arraydescr_dealloc needs to do something like.
if (self-fields == Py_None) {
print something
incref(self)
return;
}
Travis, I know you're busy right now, so this message is just so that
the archives have this info, for whenever you revisit the problem.
A long run of our code is now producing the following output:
*** Reference count error detected:
an attempt was made to deallocate 12 (d) ***
*** Reference count error detected:
an attempt was made to deallocate 12 (d) ***
etc.
Thanks to your changes it does not crash anymore, so it's not a big
deal for us. Whenever you want further details, I can try to collect
them.
Regards,
f
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy book
On 11/2/06, Travis Oliphant [EMAIL PROTECTED] wrote: Thanks for the pre-amble. Does it require pdflatex? I use ps2pdf because to generate the shaded boxes and graphics. I could probably try to do it with pdflatex and png files but I haven't tried, yet. I just tested it with ps2pdf on a document, and it worked fine. If you have problems, play with commenting (or not) the first option line to hyperref: \usepackage[ %pdftex, % needed for pdflatex In the copy I gave you it's commented out, so it /should/ work for ps2pdf. But I've had documents where I've had to uncomment it, and I think at that point they stop working with ps2pdf, and yes, you lose the usage of PostScript specials in that case (which I use for things like whole-page DRAFT watermarks). Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy.random.ranomd_integers returning only negative numbers ?
On 11/1/06, David Cournapeau [EMAIL PROTECTED] wrote: Hi, I want to generate some random integers,let's say in the range [- 2^15, 2^16]. Why doing: noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050) gives only negative numbers ? In [3]: noise = numpy.random.random_integers(- (2 ** 15), (2 * 15), 22050) In [4]: noise[noise0].shape Out[4]: (17,) In [5]: noise[noise0].shape Out[5]: (22033,) In [6]: noise = numpy.random.random_integers(-(2**15), (2 ** 15), 22050) In [7]: noise[noise0].shape Out[7]: (11006,) In [8]: noise[noise0].shape Out[8]: (11006,) In [9]: 17./22033 Out[9]: 0.00077156991785049694 In [10]: 2.0*15/2**15 Out[10]: 0.00091552734375 close enough ;) Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-user] [ANN[ pysamplerate, a library to do high quality samplerate convertion
On 11/2/06, David Cournapeau [EMAIL PROTECTED] wrote: (if those announcements are not welcome on the lists, please tell me) Frankly, if an announcement for a free Python signal processing library is not welcome on the scipy lists, I don't know where it would be ;) As a minor note though: please remember to make a note of this in the Topical Software wiki; it's good for that page to remain up to date with newly released packages and libraries. Thanks! f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] [SciPy-user] Need more comments from scientific community on python-dev
On 10/31/06, Travis Oliphant [EMAIL PROTECTED] wrote: Fernando Perez wrote: I actually worry about the same: I really would like to help, but after reading the whole discussion, I realized that the low-level details being asked and discussed are something I don't really know enough to say anything. And I don't want to sound simply saying 'Hey, Travis is great, listen to him!' to python-dev, since that (asides from looking silly) can be somewhat counter-productive. How does that sound, Travis? Is that something you think might help you, esp. since so many of us are feeling woefully underqualified to lend a useful hand in the actual discussion on python-dev? That would be great. I think a couple of things would also be useful. OK, my experience so far has been that there's a certain 'activation barrier' with Wikis, but once pages are there, people for some reason feel more comfortable filling in. So even though it's mostly a place holder, I went ahead and made this: http://www.scipy.org/ArrayInterfacePEP Hopefully as the discussion evolves, this page can be filled in with all the necessary info in one place, and it will become in a few days a solid, organized repository of all the key points in this discussion. This will be a LOT easier to refer to in the python-dev battle than randomly scattered bits of emails in the discussion thread. I'll try to spend more time on it over the next few days to fill in, but I'm pretty busy with other things as well, so hopefully others can pitch in as well. Cheers, f ps - one more thing. This guy: http://blog.vrplumber.com/ has been rewriting the OpenGL bindings using ctypes, and I've seen posts from him about numpy (in his blog). He might be able to contribute something... - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] installing numpy on Tiger
On 10/31/06, Alan Isaac [EMAIL PROTECTED] wrote: If other users will not profit from this question, please consider it OT and ignore it. But perhaps it will prove useful to some potential numpy users. I am a long time Windows user who has been happily using the Python and numpy installers. At work I need to use a Mac for a while. I installed Python 2.5 from python.org painlessly, but it looks like I'll have to compile numpy from source. Do I understand that right? I have used interpreted languages for pretty much all my computing needs, so I am hoping someone patient will point me to or lead me through the steps I need to take to successfully install numpy on this platform. This is probably a good starting point: http://www.scipy.org/Installing_SciPy/Mac_OS_X There have been numerous threads on this issue recently, so you may also want to do a bit of searching of the mailing list archives. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] glibc invalid pointer error
On 10/30/06, David Huard [EMAIL PROTECTED] wrote: Hi, I have a script that crashes, but only if it runs over 9~10 hours, with the following backtrace from gdb. The script uses PyMC, and repeatedly calls ( 100) likelihood functions written in fortran and wrapped with f2py. Numpy: 1.0.dev3327 Python: 2.4.3 This sounds awfully reminiscent of the bug I recently mentioned: http://aspn.activestate.com/ASPN/Mail/Message/numpy-discussion/3312099 We left a fresh run over the weekend, but my office mate is currently out of the office and his terminal is locked, so I don't know what the result is. I'll report shortly: we followed Travis' instructions and ran with a fresh SVN build which includes the extra warnings he added to the dealloc routines. You may want to try the same advice, perhaps with information from both of us the gurus may zero in on the problem, if indeed it is the same. Note that I'm not positive it's the same problem, and our backtraces aren't quite the same. But the rest of the scenario is similar: low-level memory crash from glibc, very long run is needed to fire the bug, potentially millions of calls to both numpy and to f2py-wrapped in-house libraries. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange and hard to reproduce crash
On 10/23/06, Travis Oliphant [EMAIL PROTECTED] wrote:
I've placed them in SVN (r3384):
arraydescr_dealloc needs to do something like.
if (self-fields == Py_None) {
print something
incref(self)
return;
}
Here is some more info. We left a long-running job over the weekend
with the prints you suggested. Oddly, something happened at the OS
level which killed our SSH connection to that machine, but the above
numpy dealloc() warning never printed (we logged this).
What did happen is that the refcount you suggested we print:
sys.getrefcount(numpy.dtype('float'))
eventually seems to have wrapped around and gone negative. I'm
attaching the log file with those print statements, the key point is
that this happens eventually:
PSVD Iteration 19
Ref count 1989827662
bar 444
PSVD Iteration 0
Ref count 2021353399
PSVD Iteration 1
Ref count 2143386207
PSVD Iteration 2
Ref count -2001245193
PSVD Iteration 3
Ref count -1915816437
PSVD Iteration 4
Ref count -1902698473
That refcount is for dtype('float') as indicated above. Is it not a
problem that this particular refcount goes negative? Eventually it
may continue increasing and hit a zero, point at which I imagine that
the bad dealloc will occur.
Are refcounts stored in signed 32-bit ints? Why? I'd have naively
expected them to be stored in unsigned longs to avoid wraparound
problems, but maybe I'm completely missing the real problem here.
We've started another run to see if we can get the actual crash to
happen, will report.
Cheers,
f
numpy_report
Description: Binary data
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] glibc invalid pointer error
On 10/30/06, Travis Oliphant [EMAIL PROTECTED] wrote: David Huard wrote: Ok, I'll update numpy and give it another try tonight. I just fixed some reference-count problems in f2py today. These were of the variety that there was a missing decref that would cause the reference count of certain often-used data-types to increase without bound and eventually wrap (to 0) in long-running processes using f2py. I suspect this is the fundamental problem in both cases. Many thanks, Travis. We're rebuilding numpy and all of our f2py-generated wrappers, and will start a new run. I'll report on the results as well. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange and hard to reproduce crash
On 10/30/06, Lisandro Dalcin [EMAIL PROTECTED] wrote: FYI, this is what is defined in Include/object.h /* PyObject_HEAD defines the initial segment of every PyObject. */ #define PyObject_HEAD \ _PyObject_HEAD_EXTRA\ Py_ssize_t ob_refcnt; \ struct _typeobject *ob_type; #define Py_INCREF(op) ( \ _Py_INC_REFTOTAL _Py_REF_DEBUG_COMMA \ (op)-ob_refcnt++) #define Py_DECREF(op) \ if (_Py_DEC_REFTOTAL _Py_REF_DEBUG_COMMA \ --(op)-ob_refcnt != 0) \ _Py_CHECK_REFCNT(op)\ else\ _Py_Dealloc((PyObject *)(op)) And '_Py_CHECK_REFCNT' macro will finally call Py_FatalError 'ob_refcnt' is a Py_ssize_t integer, so I think you will not be able to overflow it, unless in case of C code with refcounting bugs. Am I right? I think you are, and fortunately this indicates that they /did/ change to a longer data type for refcounting in newer pythons. The box where we have this problem is running 2.3 though, and obviously a runaway refcount in f2py can still die even if it's a longer data type. However, Travis mentioned he just fixed precisely a bug like that in f2py, so I'm optimistic, and I'm currently making a new test. Thanks for the info, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] pyaudio, a module to make noise from numpy arrays
On 10/27/06, David L. Goldsmith [EMAIL PROTECTED] wrote: I'm sure some others _might_ regard this as frivolous, so let me just say: Way Cool! Thanks! +1, and not frivolous at all. It's /really/ neat to be able to pull in data from standard image formats (say jpeg) into arrays to quickly do numerics on images. Adding similar capabilities to audio signals is, IMHO, a great contribution. So I heartily join the congratulation. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange and hard to reproduce crash
On 10/23/06, Travis Oliphant [EMAIL PROTECTED] wrote: Fernando Perez wrote: Hi all, two colleagues have been seeing occasional crashes from very long-running code which uses numpy. We've now gotten a backtrace from one such crash, unfortunately it uses a build from a few days ago: This looks like a reference-count problem on the data-type objects (probably one of the builtin ones is trying to be released). The reference count problem is probably hard to track down. A quick fix is to not allow the built-ins to be freed (the attempt should never be made, but if it is, then we should just incref the reference count and continue rather than die). Ideally, the reference count problem should be found, but other-wise I'll just insert some print statements if the attempt is made, but not actually do it as a safety measure. If you point me to the right place in the sources, I'll be happy to add something to my local copy, rebuild numpy and rerun with these print statements in place. I realize this is probably a very difficult problem to track down, but it really sucks to run a code for 4 days only to have it explode at the end. Right now this is starting to be a serious problem for us as we move our codes into large production runs, so I'm willing to put in the necessary effort to track it down, though I'll need some guidance from our gurus. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Strange and hard to reproduce crash
On 10/23/06, Travis Oliphant [EMAIL PROTECTED] wrote:
Fernando Perez wrote:
If you point me to the right place in the sources, I'll be happy to
add something to my local copy, rebuild numpy and rerun with these
print statements in place.
I've placed them in SVN (r3384):
[...]
Great, thanks. I'll rebuild everything from SVN.
Tracking the reference count of the built-in data-type objects should
not be too difficult. First, figure out which one is causing problems
(if you still have the gdb traceback, then go up to the
arraydescr_dealloc function and look at self-type_num and self-type).
Unfortunately we closed that gdb session.
Then, put print statements throughout your code for the reference count
of this data-type object.
Something like,
sys.getrefcount(numpy.dtype('float'))
OK, we'll log those into a file and will report after another multi-day run.
Thanks again for the help!
Cheers,
f
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Should numpy.sqrt(-1) return 1j rather than nan?
On 10/12/06, Travis Oliphant [EMAIL PROTECTED] wrote:
Why in the world does it scare you away. This makes no sense to me.
If you don't like the scipy version don't use it. NumPy and SciPy are
not the same thing.
I'd like to pitch in (again) on this issue, but I'll try to make sure
that it's clear that I'm NOT arguing about sqrt() in particular, one
way or another.
It's perfectly clear that numpy != scipy to all of us. And yet, I
think it is equally clear that the two are /very/ tightly related.
Scipy builds on top of numpy and it directly exposes a LOT of the
numpy API as scipy functions:
In [21]: import numpy as n, scipy as s
In [22]: common_names = set(dir(n)) set(dir(s))
In [23]: [getattr(n,x) is getattr(s,x) for x in common_names ].count(True)
Out[23]: 450
In [24]: len(common_names)
Out[24]: 462
That's 450 objects from numpy which are directly exposed in Scipy,
while only 12 names are in both top-level namespaces and yet are
different objects. Put another way, scipy is a direct wrap of 97% of
the numpy top-level namespace. While /we/ know they are distinct
entities, to the casual user a 97% match looks pretty close to being
the same, especially when the non-identical things are all
non-numerical:
In [27]: [x for x in common_names if getattr(n,x) is not getattr(s,x)]
Out[27]:
['pkgload',
'version',
'__config__',
'__file__',
'__all__',
'__doc__',
'show_config',
'__version__',
'__path__',
'__name__',
'info',
'test']
In [32]: n.__version__,s.__version__
Out[32]: ('1.0.dev3306', '0.5.2.dev2252')
Basically, at least for these versions, the top-level API of scipy is
a strict superset of the numpy one for all practical purposes.
I think it's fair to say that if we start sprinkling special cases
where certain objects happen to have the same name but produce
different results for the same inputs, confusion will arise.
Please note that I see a valid reason for scipy.foo != numpy.foo when
the scipy version uses code with extra features, is faster, has
additional options, etc. But as I said in a previous message, I think
that /for the same input/, we should really try to satisfy that
numpy.foo(x) == scipy.foo(x) (which is NOT the same as 'numpy.foo is scipy.foo')
within reason. Obviously the scipy version may succeed where the
numpy one fails due to better algorithms, or be faster, etc. I'm
talking about a general principle here.
I doubt I'll be able to state my point with any more clarity, so I'll
stop now. But I really believe that this particular aspect of
consistency between numpy and scipy is a /very/ important one for its
adoption in wider communities.
Best,
f
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Should numpy.sqrt(-1) return 1j rather than nan?
On 10/11/06, Travis Oliphant [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: Could sqrt(-1) made to return 1j again? Not in NumPy. But, in scipy it could. Without taking sides on which way to go, I'd like to -1 the idea of a difference in behavior between numpy and scipy. IMHO, scipy should be within reason a strict superset of numpy. Gratuitious differences in behavior like this one are going to drive us all mad. There are people who import scipy for everything, others distinguish between numpy and scipy, others use numpy alone and at some point in their life's code they do import numpy as N - import scipy as N because they start needing stuff not in plain numpy. Having different APIs and behaviors appear there is, I think, a Seriously Bad Idea (TM). Just my 1e-2j, Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Should numpy.sqrt(-1) return 1j rather than nan?
On 10/11/06, Travis Oliphant [EMAIL PROTECTED] wrote: Fernando Perez wrote: IMHO, scipy should be within reason a strict superset of numpy. This was not the relationship of scipy to Numeric. For me, it's the fact that scipy *used* to have the behavior that scipy.sqrt(-1) return 1j and now doesn't that is the kicker. That's fine, my only point was that we should really strive for consitency between the two. I think most users should be able to expect that numpy.foo(x) == scipy.foo(x) for all cases where foo exists in both. The scipy.foo() call might be faster, or take extra arguments for flexibility, and the above might only be true within floating point accuracy (since a different algorithm may be used), but hopefully functions with the same name do the same thing in both. I really think breaking this will send quite a few potential users running for the hills, and this is what I meant by 'superset'. Perhaps I wasn't clear enough. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Should numpy.sqrt(-1) return 1j rather than nan?
On 10/11/06, Travis Oliphant [EMAIL PROTECTED] wrote: Fernando Perez wrote: There are people who import scipy for everything, others distinguish between numpy and scipy, others use numpy alone and at some point in their life's code they do import numpy as N - import scipy as N because they start needing stuff not in plain numpy. Having different APIs and behaviors appear there is, I think, a Seriously Bad Idea (TM). I think the SBI is mixing numpy and scipy gratuitously (which I admit I have done in the past). I'm trying to repent Well, the problem is that it may not be so easy not to do so, esp. for new users. The fact that scipy absorbs and exposes many numpy functions makes this a particularly easy trap for anyone to fall into. The fact that even seasoned users do it should be an indicator that the 'right thing to do' is anything but obvious, IMHO. Once the dust settles on numpy 1.0, I think that the issues of how scipy plays with it, API consistency, coding best practices, etc, will need serious attention. But let's cross one bridge at a time :) Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
[Numpy-discussion] A mean docstring?
Hi all, I was updating some old Numeric code to be more 'numpythonic' (it works, but uses a lot of Numeric-style cruft), and read the a.mean() docstring that says: If axis is None, this equals: a.sum(axis, dtype) * 1.0 / product(a.shape,axis=0) Is this convoluted form of the denominator really needed? Isn't it equivalent to a.size? A few quick tests (and my understanding of numpy's new attributes) seem to indicate so: In [78]: a=rand(1,2,3,4); product(a.shape,axis=0)==a.size Out[78]: True In [79]: a=rand(2,3,4,2); product(a.shape,axis=0)==a.size Out[79]: True In [80]: a=rand(2); product(a.shape,axis=0)==a.size Out[80]: True In [81]: a=rand(2,4,4); product(a.shape,axis=0)==a.size Out[81]: True So my question is two-fold (assuming the above is correct in all cases): 1. Should the docstring be changed to simply say If axis is None, this equals: a.sum(axis, dtype) * 1.0 / a.size This is clearer to readers, and promotes the use of .size as a quick and correct way of getting an array's size. 2. Does the implementation actually used the more complicated product() approach, or does it just read .size? I'm wondering about efficiency. Regards, f - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] contribution about history of Numeric
On 10/5/06, Paul Dubois [EMAIL PROTECTED] wrote: I was reading the 'History of SciPy' page and noticed the discussion about Numeric. Here's the true story about why the various names for the original: numpy, Numeric, Numerical. I added this to the wiki page (http://www.scipy.org/History_of_SciPy), though I don't know how to make a block quote in wiki markup, so I left it without any visual clue. Thanks for the info! Best, f - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] reload and f2py
On 9/26/06, George Nurser [EMAIL PROTECTED] wrote: I'm running Python 2.3.5 with recent SVN f2py. Suppose I import an extension I have built with f2py. Then, if I edit the fortran and recompile the extension, I cannot use reload to use the modified version within the same Python session. I believe this is an problem with Python, that reload doesn't work with externally compiled extensions. As far as I know, you are correct. Is this something that can be changed in future versions of Python, or is it an inherent problem? For editing and recompiling within an ipython session is quite a good way of getting the fortran code to work, leaving aside any problems in doing the f2py interfacing. Unfortunately due to this limitation, I just restart ipython when I need to reload extension code. I'd be thrilled to know if there is any easy workaround. It's worth noting that weave is in this regard extremly ipython-friendly: since the name of the extension it loads is a hash of the source, /any/ change to the source results in a newly named extension being built and loaded. So while you get a bit of memory bloat by keeping the old, unused extensions around, the net interactive effect is that 'reloading works' for weave-generated code. One more reason why I think weave is the best thing since toasted bagels. Cheers, f - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.0b5 build failure
On 9/9/06, Neal Becker [EMAIL PROTECTED] wrote:
Sorry, actually that's today's numpy svn - not 1.0b5.
Make sure you have a clean build environment, the current SVN builds
just fine on my Dapper 6.06 box; I just rebuilt it a second ago.
Cheers,
f
ps. Here's my little build script, which I use to rebuild
numpy/scipy/matplotlib in one shot:
longs[scipy] cat makeall
#!/bin/sh
./makepkg numpy
./makepkg scipy
./makepkg matplotlib
# EOF
which uses the simple makepkg (just edit the PYTHON and PYPREFIX vars
to suit your config):
longs[scipy] cat makepkg
#!/bin/sh
PACKAGE=$1
PYTHON=python2.4
PYPREFIX=$HOME/tmp/local
#REVISION=HEAD
#REVISION=2772 # for numpy
#REVISION=2054 # for scipy
#REVISION={2006-07-06}
#svn update -r $REVISION ${PACKAGE}
svn update ${PACKAGE}
export PYTHONPATH=${PYPREFIX}/lib/${PYTHON}/site-packages:${PYTHONPATH}:
# remove existing ${PACKAGE} to make sure the build doesn't pick up
spurious things
rm -rf $PYPREFIX/lib/${PYTHON}/site-packages/${PACKAGE}
# make/install
cd ${PACKAGE}
rm -rf build
$PYTHON setup.py install --prefix=$PYPREFIX
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
[Numpy-discussion] Changing Fatal error into ImportError?
Hi all, this was mentioned in the past, but I think it fell through the cracks: Python 2.3.4 (#1, Mar 10 2006, 06:12:09) [GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2 Type help, copyright, credits or license for more information. import mwadap Overwriting info=function info at 0xb77198b4 from scipy.misc (was function info at 0xb7704bc4 from numpy.lib.utils) RuntimeError: module compiled against version 90909 of C-API but this version of numpy is 102 Fatal Python error: numpy.core.multiarray failed to import... exiting. I really think that this should raise ImportError, but NOT kill the python interpreter. If this happens in the middle of a long-running interactive session, you'll lose all of your current state/work, where a simple ImportError would have been enough to tell you that this particular module needed recompilation. FatalError should be reserved for situations where the internal state of the Python VM itself can not realistically be expected to be sane (corruption, complete memory exhaustion for even internal allocations, etc.) But killing the user's session for a failed import is a bit much, IMHO. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Changing Fatal error into ImportError?
On 8/30/06, Robert Kern [EMAIL PROTECTED] wrote: I don't see where we're calling Py_FatalError. The problem might be in Python or mwadap. Indeed, import_array() raises a PyExc_ImportError. Sorry for the noise: it looks like this was already fixed: http://projects.scipy.org/scipy/numpy/changeset/3044 since the code causing problems had been built /before/ 3044, we got the FatalError. But with modules built post-3044, it's all good (I artificially hacked the number to force the error): In [1]: import mwadap Overwriting info=function info at 0x4158402c from scipy.misc (was function info at 0x4067410c from numpy.lib.utils) --- exceptions.RuntimeError Traceback (most recent call last) RuntimeError: module compiled against version 101 of C-API but this version of numpy is 102 --- exceptions.ImportError Traceback (most recent call last) /home/fperez/research/code/mwadap-merge/mwadap/test/ipython console /home/fperez/usr/lib/python2.3/site-packages/mwadap/__init__.py 9 glob,loc = globals(),locals() 10 for name in __all__: --- 11 __import__(name,glob,loc,[]) 12 13 # Namespace cleanup /home/fperez/usr/lib/python2.3/site-packages/mwadap/Operator.py 18 19 # Our own packages --- 20 import mwrep 21 from mwadap import mwqmfl, utils, Function, flinalg 22 ImportError: numpy.core.multiarray failed to import In [2]: Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Release of 1.0b5 this weekend
On 8/29/06, Charles R Harris [EMAIL PROTECTED] wrote: Speaking of features, I wonder if more of the methods should return references. For instance, it might be nice to write something like: a.sort().searchsorted([...]) instead of making two statements out of it. +1 for more 'return self' at the end of methods which currently don't return anything (well, we get the default None), as long as it's sensible. I really like this 'message chaining' style of programming, and it annoys me that much of the python stdlib gratuitously prevents it by NOT returning self in places where it would be a perfectly sensible thing to do. I find it much cleaner to write x = foo.bar().baz(param).frob() than foo.bar() foo.baz(param) x = foo.frob() but perhaps others disagree. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Release of 1.0b5 this weekend
On 8/29/06, David M. Cooke [EMAIL PROTECTED] wrote: On Tue, 29 Aug 2006 14:03:39 -0700 Tim Hochberg [EMAIL PROTECTED] wrote: b = a.sort() # Returns a copy a.sort(out=a) # Sorts a in place a.sort(out=c) # Sorts a into c (probably just equivalent to c = a.sort() in this case since we don't want to rewrite the sort routines) Ugh. That's completely different semantics from sort() on lists, so I think it would be a source of bugs (at least, it would mean keeping two different ideas of .sort() in my head). Agreed. Except where very well justified (such as slicing returning views for memory reasons), let's keep numpy arrays similar to native lists in their behavior... Special cases aren't special enough to break the rules. and all that :) Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Release of 1.0b5 this weekend
On 8/29/06, Travis Oliphant [EMAIL PROTECTED] wrote: Hi all, Classes start for me next Tuesday, and I'm teaching a class for which I will be using NumPy / SciPy extensively. I need to have a release of these two (and hopefully matplotlib) that work with each other. Therefore, I'm going to make a 1.0b5 release of NumPy over the weekend (probably Monday), and also get a release of SciPy out as well. At that point, I'll only be available for bug-fixes to 1.0. Therefore, the next release after 1.0b5 I would like to be 1.0rc1 (release-candidate 1). What's the status of these 'overwriting' messages? planck[/tmp] python -c 'import scipy;scipy.test()' Overwriting info=function info at 0x40ba748c from scipy.misc (was function info at 0x4080409c from numpy.lib.utils) Overwriting fft=function fft at 0x430ae33c from scipy.fftpack.basic (was module 'numpy.fft' from '/home/fperez/tmp/local/lib/python2.3/site-packages/numpy/fft/__init__.pyc' from /home/fperez/tmp/local/lib/python2.3/site-packages/numpy/fft/__init__.pyc) ... I was under the impression you'd decided to quiet them out, but they seem to be making a comeback. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] request for new array method: arr.abs()
On 8/23/06, Bill Baxter [EMAIL PROTECTED] wrote: The thing that I find I keep forgetting is that abs() is a built-in, but other simple functions are not. So it's abs(foo), but numpy.floor(foo) and numpy.ceil(foo). And then there's round() which is a built-in but can't be used with arrays, so numpy.round_(foo).Seems like it would be more consistent to just add a numpy.abs() and numpy.round(). But I guess there's nothing numpy can do about it... you can't name a method the same as a built-in function, right? That's why we have numpy.round_() instead of numpy.round(), no? [...goes and checks] Oh, you *can* name a module function the same as a built-in. Hmm... so then why isn't numpy.round_() just numpy.round()? Is it just so from numpy import * won't hide the built-in? Technically numpy could simply have (illustrated with round, but works also with abs) round = round_ and simply NOT include round in the __all__ list. This would make numpy.round(x) work (clean syntax) while from numpy import * would not clobber the builtin round. That sounds like a decent solution to me. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Interesting memory leak
This leak is caused by add_docstring, but it's supposed to leak. I wonder if there's a way to register some kind of on-exit handler in Python so that this can also be cleaned up? import atexit atexit.register(your_cleanup_function) whose api is no args on input: def your_cleanup_function(): do_whatever... You could use here a little extension function which goes in and does the necessary free() calls on a pre-stored list of allocated pointers, if there's more than one (I don't really know what's going on here). Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Creating and reshaping fortran order arrays
On 8/14/06, Travis Oliphant [EMAIL PROTECTED] wrote: Travis Oliphant wrote: However, you can use the ndarray creation function itself to do what you want: a = ndarray(shape=(2,2), dtype=int32, buffer=str, order='F') This will use the memory of the string as the new array memory. Incidentally, the new array will be read-only. But, you can fix this in two ways: 1) a.flags.writeable = True Sweet! We now finally have mutable strings for Python: In [2]: astr = '\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00' In [4]: a = N.ndarray(shape=(2,2), dtype=N.int32, buffer=astr, order='F') In [5]: astr Out[5]: '\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00' In [6]: a.flags.writeable = True In [7]: a Out[7]: array([[0, 2], [1, 3]]) In [8]: a[0] = 1 In [9]: astr Out[9]: '\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x03\x00\x00\x00' Guido's going to kill you on Thursday, you know ;) f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Version numbers again
On 7/27/06, Francesc Altet [EMAIL PROTECTED] wrote: Travis, Speaking on what we regularly do, I would choose a 1.0b2.svn_version for the trunk version. This is a way to say people: Hey, you are using a version that will be the 1.0b2 in the future.. Of course, the same meaning can be achieved if you interpret that a X.Y.Z.svn_version means something like post-X.Y.Z. I think all depends on interpretation. I personally prefer the first option (i.e. start using a future release number in SVN), rather than the second (i.e. using a past release version in SVN) because I tend to find it slightly less confusing. Just as a reference, that's also what I do with ipython: planck[homepage] ip Python 2.3.4 (#1, Feb 2 2005, 12:11:53) Type copyright, credits or license for more information. IPython 0.7.3.svn -- An enhanced Interactive Python. This means 'what will be 0.7.3, currently from svn'. I don't actually tack the revision number, only the 'svn' indicator, but it's the same idea. Since ipython's dev pace is slower, I don't worry so much about individual numbers, but this lets me know easily if someone is reporting a bug from running off SVN or an official release. Cheers, f - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Please comment on SVN versioning convention
On 7/24/06, Travis Oliphant [EMAIL PROTECTED] wrote: Andrew Straw has emphasized that the current strategy of appending the SVN version number to development versions of the SVN tree makes it hard to do version sorting. His proposal is to not change the version number until the first beta comes out. I have to say that being able to quickly see the actual SVN revision number in the version helps a LOT in tracking down problems. Just look at how many posts on the list start with the canonical In [4]: import numpy,scipy In [5]: numpy.__version__ Out[5]: '0.9.9.2803' In [6]: scipy.__version__ Out[6]: '0.5.0.2079' printouts before discussing the problem. I don't really feel strongly about the issue, but if you change this, then please add a __revision__ attribute as well, so that this information can be quickly asked for in pure python (we don't want to bother newbies with obscure SVN lingo, esp. if they got their install from someone else). Cheers, f - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] fast way of doing cross-multiplications ?
On 7/18/06, Tim Hochberg [EMAIL PROTECTED] wrote: Eric Emsellem wrote: thanks for the tips. (indeed your add.reduce is correct: I just wrote this down too quickly, in the script I have a sum included). And yes you are right for the memory issue, so I may just keep the loop in and try to make it work on a fast PC...(or use parallel processes) (is sum different than add.reduce?) thanks again to both Bill Baxter and Perry Greenfield for their fast (and helpful!) answers. I just wanted to add that there are faster, but considerably complicated ways to attack this class of problems. The one I've looked at in the past was the fast multipole method and I believe there are others. I'm not sure whether these can be implemented efficiently in numpy, but you may want to take a look into this kind of more sophisticated/complicated approach if brute forcing the calculation doesn't work. Indeed, FMM is the best known method that can turn this O(n^2) problem into O(n*log(n)). As Tim indicates, there is no easy way out of this one. Incidentally, there is a talk about FMM on the scipy'06 schedule, in case you are going to attend. An alternative approach to FMM (conceptually similar in some ways) is described here: http://amath.colorado.edu/faculty/fperez/talks/0604_sanum_mwadap.pdf Unfortunately this isn't exactly a half-hour optimization effort, as the required machinery is pretty heavy duty. And yes, this has all been done in python and runs on current numpy/scipy (though it has Fortran, C and C++ sprinkled as needed). Cheers, f - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
[Numpy-discussion] 64-bit build woes
Hi all, I received this message today from a collaborator. I don't have direct access to this box, but he posted fairly detailed logs. Has anyone seen a similar issue with current code? If not, I'll try to track down further with him what the problem may actually be. Thanks for any help, f Original message: We are having trouble with scipy (not)compiling on the 64-bit machine, and it seems to be related to the intp type. I put the log file and connected files at http://www.math.ohiou.edu/~mjm/pickup/scipy.log http://www.math.ohiou.edu/~mjm/pickup/fortranobject.c http://www.math.ohiou.edu/~mjm/pickup/fortranobject.h The relevant parts are at the end of scipy.log: creating build/temp.linux-x86_64-2.3/Lib/fftpack/src compile options: '-Ibuild/src.linux-x86_64-2.3 -I/usr/lib64/python2.3/site-packages/numpy/core/include -I/usr/include/python2.3 -c' gcc: Lib/fftpack/src/drfft.c gcc: Lib/fftpack/src/zfft.c gcc: build/src.linux-x86_64-2.3/fortranobject.c build/src.linux-x86_64-2.3/fortranobject.c: In function `PyFortranObject_New': build/src.linux-x86_64-2.3/fortranobject.c:55: error: syntax error before intp build/src.linux-x86_64-2.3/fortranobject.c:60: error: syntax error before intp build/src.linux-x86_64-2.3/fortranobject.c: In function `fortran_getattr': build/src.linux-x86_64-2.3/fortranobject.c:179: error: syntax error before intp build/src.linux-x86_64-2.3/fortranobject.c: In function `fortran_setattr': build/src.linux-x86_64-2.3/fortranobject.c:243: error: syntax error before ')' token build/src.linux-x86_64-2.3/fortranobject.c:245: error: syntax error before ')' token ... I don't see intp in fortranobject.c, so it must be included from elsewhere. - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Scalars, comparisons and indexing. Is this a bug?
On 7/14/06, Bill Baxter [EMAIL PROTECTED] wrote: I believe that's the problem that the indexing PEP from Travis is supposed to address: http://www.python.org/dev/peps/pep-0357/ So I think there's not much anyone can do about it untill the PEP is accepted and implemented. Maybe you can cast to int? In [34]: (1,2)[int(a[0]==b)] Yup, that's the workaround I'm using. I was just wondering if comparisons between array scalars shouldn't return /true/ booleans (which can be used as indices) instead of array scalar booleans. I realize that the __index__ support in 2.5 will make this point moot (the PEP you point to), but perhaps this particular one can actually be fixed today, for all users of python pre-2.5. However, I haven't really wrapped my head enough around all the subtleties of array scalars to know whether 'fixing' this particular problem will introduce other, worse ones. Cheers, f - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] bdist_rpm failure
On 7/11/06, Travis Oliphant [EMAIL PROTECTED] wrote:
Make sure you get rid of the MANIFEST file in the source directory
before trying to run sdist or bdist_rpm. The MANIFEST file is not being
deleted when it is dated...
Since both numpy/scipy have a MANIFEST.in, this bit of code (from
ipython's setup.py) is probably a good idea to put in:
# BEFORE importing distutils, remove MANIFEST. distutils doesn't properly
# update it when the contents of directories change.
if os.path.exists('MANIFEST'): os.remove('MANIFEST')
I committed this as I don't see any problem with it (ipython has used
it for years). Feel free to revert if you object.
I also added
#!/usr/bin/env python
to both of them. They were set as executable scripts but without a
proper shebang line, causing the interesting mis-behavior of being
treated like shell scripts. Which means the first
import os
line would simply give you a weird-looking cross-hairs cursor, and a
gigantic file called 'os' sitting on your directory once you clicked.
Why? That's the ImageMagick 'import' command, which takes a
screenshot in X bitmap format :)
This one bit me a few times, so I decided to also fix it, but feel
free to revert if you see a good reason for the existing behavior.
Cheers,
f
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] I think something broke badly today in numpy/scipy...
On 7/8/06, Travis Oliphant [EMAIL PROTECTED] wrote: Hey Fernando. I think I found the problem. It was the same problem causing the BFGS test to fail in SciPy. [...] The problem should be fixed in SVN. Confirmed. With: numpy version: 0.9.9.2788 scipy version: 0.5.0.2057 I checked on two machines (home/office) and all is good now. I only had to make a tiny change to my hand-written extensions for the new C API names, and all checks OK now. Very sorry... Considering how much you've given us, I don't really think you even have the /right/ to apologize to anyone ;) Many thanks for such a remarkable fix, given my impossibly vague bug report. You rock. f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] NumPy C-API now has prefixes
On 7/7/06, Travis Oliphant [EMAIL PROTECTED] wrote: I just committed a big change to the NumPy SVN (r2773-r2777) which adds the prefix npy_ or NPY_ to all names not otherwise pre-fixed. There is also a noprefix.h header that allows you to use the names without the prefixes defined, as before Plus: 1) The special FLAG names with _FLAGS now have the _FLAGS removed 2) The PY_ARRAY_TYPES_PREFIX is ignored. 3) The tMIN/tMAX macros are removed 4) MAX_DIMS -- NPY_MAXDIMS 5) OWN_DATA -- NPY_OWNDATA Make sure scipy builds after these, I think I just saw it not build with 'OWN_DATA' errors. Maybe I just caught you in-between commits... f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
[Numpy-discussion] I think something broke badly today in numpy/scipy...
Hi all, I updated earlier today (about 4 hours ago) numpy/scipy SVN, and all of a sudden my codes broke left and right. Backing off to In [3]: numpy.__version__ Out[3]: '0.9.9.2737' In [4]: scipy.__version__ Out[4]: '0.5.0.2044' things are OK again. I am really sorry not to be able to provide a narrow test case, but I am utterly swamped right now, and need to finish a number of things. Given how bracketing this particular problem takes about 1/2 hour in recompilations alone for each revision, I really can't do it right now. There is no exception, no traceback, it's just that various numerical algorithms fail to converge. This code is complex and it uses numpy, scipy and weave, so the problem could be anywhere. Again, my apologies for the super-vague report. But perhaps knowing that the change is fairly recent, someone may have an eureka moment :) If nobody finds anything obvious, I might be able to spend some serious debugging time next week on this, but for now I'll just retreat to a working version and try to finish my things. Cheers, f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Time for beta1 of NumPy 1.0
On 6/30/06, Scott Ransom [EMAIL PROTECTED] wrote: +1 for float64 for me as well. +1 for float64 I have lots of code overriding the int defaults by hand which were giving me grief with hand-written extensions (which were written double-only for speed reasons). I'll be happy to clean this up. I completely understand Travis' concerns about backwards compatibility, but frankly, I think that right now the quality and community momentum of numpy is already enough that it will carry things forward. People will suffer a little during the porting days, but they'll be better off in the long run. I don't think we should undrestimate the value of eternal happiness :) Besides, decent unit tests will catch these problems. We all know that every scientific code in existence is unit tested to the smallest routine, so this shouldn't be a problem for anyone. Cheers, f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Time for beta1 of NumPy 1.0
On 6/30/06, Sasha [EMAIL PROTECTED] wrote: On 6/30/06, Fernando Perez [EMAIL PROTECTED] wrote: ... Besides, decent unit tests will catch these problems. We all know that every scientific code in existence is unit tested to the smallest routine, so this shouldn't be a problem for anyone. Is this a joke? Did anyone ever measured the coverage of numpy unittests? I would be surprized if it was more than 10%. Of course it's a joke. So obviously one for anyone who knows the field, that the smiley shouldn't be needed (and yes, I despise background laughs on television, too). Maybe a sad joke, given the realities of scientific computing, and maybe a poor joke, but at least an attempt at humor. Cheers, f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] fread codes versus numpy types
On 6/28/06, Robert Kern [EMAIL PROTECTED] wrote: The Capitalized versions are actually old typecodes for backwards compatibility with Numeric. In recent development versions of numpy, they are no longer exposed except through the numpy.oldnumeric compatibility module. A decision was made for numpy to use the actual width of a type in its name instead of the width of its component parts (when it has parts). Code in scipy which still requires actual string typecodes is a bug. Please report such cases on the Trac: http://projects.scipy.org/scipy/scipy Well, an easy way to make all those poke their ugly heads in a hurry would be to remove line 32 in scipy's init: longs[Lib] grep -n oldnum *py __init__.py:31:import numpy.oldnumeric as _num __init__.py:32:from numpy.oldnumeric import * If we really want to push for the new api, I think it's fair to change those two lines by simply from numpy import oldnumeric so that scipy also exposes oldnumeric, and let all deprecated names be hidden there. I just tried this change: Index: __init__.py === --- __init__.py (revision 2012) +++ __init__.py (working copy) @@ -29,9 +29,8 @@ # Import numpy symbols to scipy name space import numpy.oldnumeric as _num -from numpy.oldnumeric import * -del lib -del linalg +from numpy import oldnumeric + __all__ += _num.__all__ __doc__ += Contents and scipy's test suite still passes (modulo the test_cobyla thingie Nils is currently fixing, which is not related to this). Should I apply this patch, so we push the cleaned-up API even a bit harder? Cheers, f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] fread codes versus numpy types
On 6/28/06, David M. Cooke [EMAIL PROTECTED] wrote: On Wed, 28 Jun 2006 11:22:38 -0600 Fernando Perez [EMAIL PROTECTED] wrote: Should I apply this patch, so we push the cleaned-up API even a bit harder? Yes please. I think all the modules that still use the oldnumeric names actually import numpy.oldnumeric themselves. Done, r2017. I also committed the simple one-liner: Index: weave/inline_tools.py === --- weave/inline_tools.py (revision 2016) +++ weave/inline_tools.py (working copy) @@ -402,7 +402,7 @@ def compile_function(code,arg_names,local_dict,global_dict, module_dir, compiler='', - verbose = 0, + verbose = 1, support_code = None, headers = [], customize = None, from a discussion we had a few weeks ago, I'd forgotten to put it in. I did it as a separate patch (r 2018) so it can be reverted separately if anyone objects. Cheers, f Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Setuptools leftover junk
On 6/28/06, David M. Cooke [EMAIL PROTECTED] wrote: Done. I've also added a 'setupegg.py' module that wraps running 'setup.py' with an import of setuptools (it's based on the one used in matplotlib). easy_install still works, also. You beat me to it :) However, your patch has slightly different semantics from mine: if bdist_egg fails to import, the rest of setuptools is still used. I don't know if that's safe. My patch would consider /any/ failure in the setuptools imports as a complete setuptools failure, and revert out to basic distutils. Let me know if you want me to put in my code instead, here's a patch from my code against current svn (after your patch), in case you'd like to try it out. Cheers, f Index: core.py === --- core.py (revision 2701) +++ core.py (working copy) @@ -1,20 +1,30 @@ - import sys from distutils.core import * -if 'setuptools' in sys.modules: -have_setuptools = True -from setuptools import setup as old_setup -# easy_install imports math, it may be picked up from cwd -from setuptools.command import develop, easy_install +# Don't pull setuptools in unless the user explicitly requests by having it +# imported (Andrew's trick). +have_setuptools = 'setuptools' in sys.modules + +# Even if setuptools is in, do a few things carefully to make sure the version +# is recent enough to have everything we need before assuming we can proceed +# using setuptools throughout +if have_setuptools: try: -# very old versions of setuptools don't have this +from setuptools import setup as old_setup +# very old setuptools don't have this from setuptools.command import bdist_egg +# easy_install imports math, it may be picked up from cwd +from setuptools.command import develop, easy_install except ImportError: +# Any failure here is probably due to an old or broken setuptools +# leftover in sys.modules, so treat it as if it simply weren't +# available. have_setuptools = False -else: + +# If setuptools was flagged as unavailable due to import problems, we need the +# basic distutils support +if not have_setuptools: from distutils.core import setup as old_setup -have_setuptools = False from numpy.distutils.extension import Extension from numpy.distutils.command import config Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Recarray attributes writeable
On 6/17/06, Francesc Altet [EMAIL PROTECTED] wrote: However, I think that this has its utility, specially when accessing to nested fields (see later). In addition, I'd suggest introducing a special accessor called, say, 'fields' in order to access the fields themselves and not the attributes. For example, if you want to access the 'strides' attribute, you can do it in the usual way: import numpy tr=numpy.recarray(10, formats='i4,f8,f8', names='id,ra,strides') tr.strides (20,) but, if you want to access *field* 'strides' you could do it by issuing: tr.fields.strides repr of field accessor object (shape, type...) tr.fields.strides[:] array([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]) [...] +1 I meant to write exactly the same thing, but was too lazy to do it :) Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Array interface updated to Version 3
On 6/16/06, Travis Oliphant [EMAIL PROTECTED] wrote: There is talk of ctypes supporting the new interface which is a worthy development. Please encourage that if you can. That would certainly be excellent, esp. given how ctypes is slated to be officially part of python 2.5. I think it would greatly improve the interoperability landscape for python if the out-of-the-box toolset had proper access to numpy arrays. Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy segv on OpenBSD
On 6/15/06, Damien Miller [EMAIL PROTECTED] wrote: David M. Cooke wrote: Can you update to the latest svn? We may have fixed it already: valgrind is showing up nothing for me. Ok, but dumb question: how do I check out the SVN trunk? Sourceforge lists details for CVS only... (unless I'm missing something) http://scipy.org/Developer_Zone Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Re: [Numpy-discussion] Getting rid of annoying weave nag
On 6/9/06, David M. Cooke [EMAIL PROTECTED] wrote: This difference is so dramatic that I think a message is justified (absent a proper logging framework). It's helpful to know that the time is going into c++ compilation, and not your code hanging for 30 seconds. Ok, I'll give you that one :-) It's the other 1000 uses of print that I'm concerned about. inline_tools.compile_function takes a verbose flag, though, which eventually gets passed to build_tools.build_extension (which I believe does all the compiling for weave). It's probably more reasonable to have inline_tools.compile_function default to verbose=1 instead of 0, then build_extension will print 'Compiling code...' (that should be changed to mention weave). I failed to mention that I agree with you: the proper solution is to use logging for this. For now I'll commit the strict-prototypes fix, and if I find myself with a lot of spare time, I'll try to clean things up a little bit to use logging (there's already a logger instance running in there). Cheers, f ___ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
