Re: 2d array slicing problem
Tune Kamae (sent by Nabble.com) wrote: > I am thinking to upgrade my desktop to 64bit cpu with 16GB memory to handle > large astronomical images and data. I wonder if > 1) the latest numarry (besides 2d slicing) has been tested with one or more > 64 bit CPU and Linux distributions Certainly. > 2) with 64 bit address space, will numarray be able to handle larger arrays > and matrices (many 3d-arrays 100x100x100 and matrices 50k x 50k)? > (with 32 bit CPU I was limited by the memory.) > > I would appreciate knowing about your experience. You'll probably want to ask on the appropriate mailing list[1], but since you ask here, I will say that there are limitations that prevent numarray from fully utilizing 64-bit systems. numarray uses the Python buffer interface which is addressed by 32-bit integers even on 64-bit platforms[2]. There has been some work on numarray's replacement, scipy_core, to address this deficiency, but more work needs to be done and more volunteers with 64-bit systems are needed. 100x100x100 arrays should work with numarray; 5x5 perhaps not. You could always try and let us know. [1] http://lists.sourceforge.net/lists/listinfo/numpy-discussion [2] http://permalink.gmane.org/gmane.comp.python.numeric.general/2690 -- Robert Kern [EMAIL PROTECTED] "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
Re: 2d array slicing problem
I am thinking to upgrade my desktop to 64bit cpu with 16GB memory to handle large astronomical images and data. I wonder if 1) the latest numarry (besides 2d slicing) has been tested with one or more 64 bit CPU and Linux distributions 2) with 64 bit address space, will numarray be able to handle larger arrays and matrices (many 3d-arrays 100x100x100 and matrices 50k x 50k)? (with 32 bit CPU I was limited by the memory.) I would appreciate knowing about your experience. Tune Kamae, SLAC/KIPAC, Stanford University Sent from the Python - python-list forum at Nabble.com. -- http://mail.python.org/mailman/listinfo/python-list
Re: 2d array slicing problem
thanks, i ran it -- the only difference i got was the numarray version: 1.1.1 on the 686, and 1.3.3 on the 64bit... but i wouldn't have thought that would make too much difference. -- http://mail.python.org/mailman/listinfo/python-list
Re: 2d array slicing problem
Do you have a simple program that demonstrates the problem? I have an x86 machine with Python 2.3, and an x86_64 machine with Python 2.4 available. I wrote a simple test program which performs a slice operation, but it behaves the same on both platforms. Here's the program: # import numarray, sys, os print "python: ", sys.version_info print "numarray:", numarray.__version__ print "CPU: ", os.uname()[-1] n = numarray.arange(10) o = numarray.outerproduct(n,n) p = o[3:7,6:10].copy() q = numarray.outerproduct([3,4,5,6], [6,7,8,9]) print "Success: ", numarray.alltrue(p.ravel() == q.ravel()) # Here are the two results I gathered: python: (2, 4, 1, 'final', 0) numarray: 1.3.3 CPU: x86_64 Success: True python: (2, 3, 2, 'final', 0) numarray: 0.9 CPU: i686 Success: 1 Jeff pgpBMzxSrksM3.pgp Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: 2d array slicing problem
jeg wrote: > dear all, > > i'm an astronomer working with 2d images -- 2d numarrays. i have a > script which basically does some operations on some images, and one of > the first steps is to find a galaxy on an image (at, say, a known x,y > coord), and create a sub-image by slicing out part of the larger array > to create a more managable smaller one (e.g. 50x50 pixels from a > 1000x1000 array). > > i was running this fine under python 2.3.4, but due to the slowness of > my machine, moved to a speedy 64-bit linux box running version 2.4.2. > arrays were no longer being sliced in the correct way, for example: > > B = A[3:7,6:10] > > generates different Bs if i run an identical script on the same image > on the different machines. > > note: we deal with FITS format images, which are read into numarrays > using the pyfits.py module. does anyone know any obvious reason this > should be happening? Not without having the data and the machines in front of me. You'll get better luck asking on numarray's mailing list. -- Robert Kern [EMAIL PROTECTED] "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter -- http://mail.python.org/mailman/listinfo/python-list
