Re: [Numpy-discussion] Can not update a submatrix
Thanks for your explanation. It explains also the following: >>> R = N.arange(9).reshape(3,3) >>> ax = [0,2] >>> U = R[ax,:] >>> U array([[0, 1, 2], [6, 7, 8]]) >>> U[:] = 100 >>> U array([[100, 100, 100], [100, 100, 100]]) >>> R# No change since U is a copy array([[0, 1, 2], [3, 4, 5], [6, 7, 8]]) >>> R[ax,:] = 100 # Works through __setitem__ >>> R array([[100, 100, 100], [ 3, 4, 5], [100, 100, 100]]) Nadav On Wed, 2008-01-30 at 13:35 -0700, Timothy Hochberg wrote: > > > > On Jan 30, 2008 12:43 PM, Anne Archibald <[EMAIL PROTECTED]> > wrote: > > On 30/01/2008, Francesc Altet <[EMAIL PROTECTED]> wrote: > > A Wednesday 30 January 2008, Nadav Horesh escrigué: > > > In the following piece of code: > > > >>> import numpy as N > > > >>> R = N.arange(9).reshape(3,3) > > > >>> ax = [1,2] > > > >>> R > > > > > > array([[0, 1, 2], > > >[3, 4, 5], > > >[6, 7, 8]]) > > > > > > >>> R[ax,:][:,ax] = 100 > > > >>> R > > > > > > array([[0, 1, 2], > > >[3, 4, 5], > > >[6, 7, 8]]) > > > > > > Why R is not updated? > > > > Because R[ax] is not a view of R, but another copy of the > original > > object (fancy indexing does return references to different > objects). > > In order to get views, you must specify only a slice of the > original > > array. For example: > > > > This is not exactly correct. > > > [...a fine explanation by Anne...] > > Another way to look at this is note that: > > > R[ax,:][:,ax] = 100 > > > > is translated to: > > > > R.__getitem__((ax, :)).__setitem__((:,ax), 100) > > > > '__setitem__' is capable of setting values using fancy indexing, but > as has been noted, '__getitem__' makes a copy of R when using fancy > indexing, and therefore the original does not get modified. > > [Technically, ':' gets translated to slice(None,None,None), but I > didn't want to write that out]. > > -- > . __ > . |-\ > . > . [EMAIL PROTECTED] > > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] blcr checkpoint/restart
Hi numeric processing fans. I'm pleased to report that you can now have convenient checkpoint/restart, at least if you are running fedora linux. Berkeley Lab Checkpoint/Restart for Linux (BLCR) http://ftg.lbl.gov/CheckpointRestart/CheckpointDownloads.shtml What I've done is: 1) built 2 rpm packages for blcr. First installs everything except kernel module. 2nd is the kernel module, packaged for use with akmods, which will be (I think) the kmod format of choice in rpmfusion. 2) made a python ctypes module to use it. This allows your python program to checkpoint itself. You will need a couple of packages from livna development to use this: rpm -q --whatprovides kmodtool kmodtool-1-7.fc8.noarch rpm -q --whatprovides akmods akmods-0.2.1-1.fc8.noarch You can find this stuff here: http://livna-dl.reloumirrors.net/fedora/development/SRPMS/ I grabbed the development srpms from livna and built them for my F8 machine. My stuff is here: https://nbecker.dyndns.org/RPM/blcr_mod.py https://nbecker.dyndns.org/RPM/blcr-0.6.4-1.src.rpm https://nbecker.dyndns.org/RPM/blcr-kmod-0.6.4-2.fc8.src.rpm ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Can not update a submatrix
A Wednesday 30 January 2008, Timothy Hochberg escrigué: > [...a fine explanation by Anne and Timothy...] Ok. As it seems that this subject has interest enough, I went ahead and created a small document about views vs copies at: http://www.scipy.org/Cookbook/ViewsVsCopies I think it resumes what has been said in this thread, but feel free to update it if you find some inaccuracy or want to complete it more. Although perhaps this is more a FAQ than a recipe, I've added it to the SciPy cookbook because I find it a bit long for a FAQ entry (but I may be wrong). In fact, I feel that the cookbook would need some reorganization, because there are too many different subjects in NumPy/SciPy section of http://www.scipy.org/Cookbook. One first action, IMHO, would be to create separate NumPy/SciPy FAQs and Cookbooks. It's my impression that NumPy is a much more general tool than SciPy with much less requeriments to install, test and use it, and having such a monolithic merge of NumPy and SciPy cookbok could discourage many potential NumPy users that might see NumPy as 'too scientific'. After that, I'd introduce a new subsection called 'Advanced topics' in both NumPy & SciPy FAQ/Cookbook. For example, currently there exists things like "SphericalBesselZeros", "Savitzky Golay filtering of data" in the first section of the cookbook, which is clearly a bit overhelming for the naive user. I'd like to hear some comments about this proposal from the community. At any rate, I've placed the "ViewsVsCopies" entry under a new section that I created in the cookbook, that I called "Advanced topics", but I'm open to suggestions on a better name/place. Cheers, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-" ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] searchsorted bug
Hi, The following gives the wrong answer: In [2]: A = array(['a','aa','b']) In [3]: B = array(['d','e']) In [4]: A.searchsorted(B) Out[4]: array([3, 0]) The answer should be [3,3]. I've come across this while trying to come up with an ismember function which works for strings (setmember1d doesn't seems to assume numerical arrays). Thanks, James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] [numpy-discussion] searchsorted bug
Hi, OK, i'm using: In [6]: numpy.__version__ Out[6]: '1.0.3' Should I try the development version? Which version of numpy would people generally recommend? James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
works fine for me. In [33]: A = numpy.array(['a','aa','b']) In [34]: B = numpy.array(['d','e']) In [35]: A.searchsorted(B) Out[35]: array([3, 3]) In [36]: numpy.__version__ Out[36]: '1.0.5.dev4567' L. On 1/31/08, James Philbin <[EMAIL PROTECTED]> wrote: > > Hi, > > The following gives the wrong answer: > > In [2]: A = array(['a','aa','b']) > > In [3]: B = array(['d','e']) > > In [4]: A.searchsorted(B) > Out[4]: array([3, 0]) > > The answer should be [3,3]. I've come across this while trying to come > up with an ismember function which works for strings (setmember1d > doesn't seems to assume numerical arrays). > > Thanks, > James > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > -- Lorenzo Bolla [EMAIL PROTECTED] http://lorenzobolla.emurse.com/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Hmmm. Just downloaded and installed 1.0.4 and i'm still getting this error. Are you guys using the bleeding edge version or the official 1.0.4 tarball from the webpage? James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] [numpy-discussion] searchsorted bug
Hi, The following gives the wrong answer: In [2]: A = array(['a','aa','b']) In [3]: B = array(['d','e']) In [4]: A.searchsorted(B) Out[4]: array([3, 0]) The answer should be [3,3]. I've come across this while trying to come up with an ismember function which works for strings (setmember1d doesn't seems to assume numerical arrays). Thanks, James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
It works fine also for me (numpy 1.04 gentoo linux on amd64) Nadav On Thu, 2008-01-31 at 15:51 +0100, lorenzo bolla wrote: > works fine for me. > > > In [33]: A = numpy.array(['a','aa','b']) > > In [34]: B = numpy.array(['d','e']) > > In [35]: A.searchsorted(B) > Out[35]: array([3, 3]) > > In [36]: numpy.__version__ > Out[36]: '1.0.5.dev4567' > > > > > > L. > > > > > On 1/31/08, James Philbin <[EMAIL PROTECTED]> wrote: > > Hi, > > The following gives the wrong answer: > > In [2]: A = array(['a','aa','b']) > > In [3]: B = array(['d','e']) > > In [4]: A.searchsorted(B) > Out[4]: array([3, 0]) > > The answer should be [3,3]. I've come across this while trying > to come > up with an ismember function which works for strings > (setmember1d > doesn't seems to assume numerical arrays). > > Thanks, > James > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > > > > -- > Lorenzo Bolla > [EMAIL PROTECTED] > http://lorenzobolla.emurse.com/ > > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
I use a dev version (1.0.5.dev4567). L. On 1/31/08, James Philbin <[EMAIL PROTECTED]> wrote: > > Hmmm. Just downloaded and installed 1.0.4 and i'm still getting this > error. Are you guys using the bleeding edge version or the official > 1.0.4 tarball from the webpage? > > James > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > -- Lorenzo Bolla [EMAIL PROTECTED] http://lorenzobolla.emurse.com/ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Hi, Just tried with numpy from svn and still get this problem: >>> import numpy >>> numpy.__version__ '1.0.5.dev4763' >>> A = numpy.array(['a','aa','b']) >>> B = numpy.array(['d','e']) >>> A.searchsorted(B) array([3, 0]) I guess this must be a platform-dependent bug. I'm running python version: Python 2.5 (r25:51908, Nov 6 2007, 15:55:44) [GCC 4.1.2 20070925 (Red Hat 4.1.2-27)] on linux2 I'm on an Intel Xeon E5345. Any help would be much appreciated. Thanks, James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
No problem for me (also a svn version) : Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11) [GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2 >>> import numpy >>> A = numpy.array(['a','aa','b']) >>> B = numpy.array(['d','e']) >>> A.searchsorted(B) array([3, 3]) Matthieu 2008/1/31, lorenzo bolla <[EMAIL PROTECTED]>: > > I use a dev version (1.0.5.dev4567). > L. > > > On 1/31/08, James Philbin <[EMAIL PROTECTED]> wrote: > > > > Hmmm. Just downloaded and installed 1.0.4 and i'm still getting this > > error. Are you guys using the bleeding edge version or the official > > 1.0.4 tarball from the webpage? > > > > James > > ___ > > Numpy-discussion mailing list > > Numpy-discussion@scipy.org > > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > > > > > -- > Lorenzo Bolla > [EMAIL PROTECTED] > http://lorenzobolla.emurse.com/ > > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
I do get the problem with a recent(ish) svn, on OS X 10.5.1, python 2.5.1 (from python.org): In [76]: A = array(['a','aa','b']) In [77]: B = array(['d','e']) In [78]: A.searchsorted(B) Out[78]: array([3, 0]) In [79]: numpy.__version__ Out[79]: '1.0.5.dev4722' ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] matrix <-> ndarray bug
I don't know, maybe it's already fixed in more recent versions?
>>> from numpy import *
>>> a=mat('1 2')
>>> b = asfarray(a).flatten()
>>> print b[0]
[[ 1. 2.]]
# ^^ I expected getting 1.0 here
>>> numpy.version.version
'1.0.3'
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Hi, With my system running x86_64 SUSE10.0 AMD opteron: Under Python 2.5.1 (Python 2.5.1 -r251:54863) and numpy 1.0.4 (download of released version) I have the same bug. Under Python 2.4.1 (May 25 2007, 18:41:31) and numpy 1.0.3 I have no problem. Perhaps a 32/64 bit problem? Bruce On Jan 31, 2008 9:17 AM, Matthieu Brucher <[EMAIL PROTECTED]> wrote: > No problem for me (also a svn version) : > > Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11) > [GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2 > > >>> import numpy > > >>> A = numpy.array(['a','aa','b']) > >>> B = numpy.array(['d','e']) > >>> A.searchsorted(B) > array([3, 3]) > > Matthieu > > 2008/1/31, lorenzo bolla <[EMAIL PROTECTED]>: > > > > I use a dev version (1.0.5.dev4567). > > L. > > > > > > On 1/31/08, James Philbin <[EMAIL PROTECTED]> wrote: > > > > > Hmmm. Just downloaded and installed 1.0.4 and i'm still getting this > > > error. Are you guys using the bleeding edge version or the official > > > 1.0.4 tarball from the webpage? > > > > > > James > > > ___ > > > Numpy-discussion mailing list > > > Numpy-discussion@scipy.org > > > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > > > > > > > > > > > > -- > > Lorenzo Bolla > > [EMAIL PROTECTED] > > http://lorenzobolla.emurse.com/ > > ___ > > Numpy-discussion mailing list > > Numpy-discussion@scipy.org > > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > > > > > > > -- > French PhD student > Website : http://matthieu-brucher.developpez.com/ > Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 > LinkedIn : http://www.linkedin.com/in/matthieubrucher > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Am Donnerstag, 31. Januar 2008 15:35:25 schrieb James Philbin: > The following gives the wrong answer: > > In [2]: A = array(['a','aa','b']) > > In [3]: B = array(['d','e']) > > In [4]: A.searchsorted(B) > Out[4]: array([3, 0]) > > The answer should be [3,3]. Heh, I got both answers in the same session (not 100% reproducable, but several times already)! In [1]: %cpaste Pasting code; enter '--' alone on the line to stop. :>>> import numpy :>>> A = numpy.array(['a','aa','b']) :>>> B = numpy.array(['d','e']) :>>> print A.searchsorted(B) :>>> print numpy.__version__ :-- [3 3] 1.0.5.dev4420 In [2]: In [3]: print A.searchsorted(B) [3 3] In [4]: print A.searchsorted(B) [3 3] In [5]: print A.searchsorted(B) [3 3] In [6]: %cpaste Pasting code; enter '--' alone on the line to stop. :>>> import numpy :>>> A = numpy.array(['a','aa','b']) :>>> B = numpy.array(['d','e']) :>>> print A.searchsorted(B) :>>> print numpy.__version__ :-- [3 0] 1.0.5.dev4420 -- Ciao, / / /--/ / / ANS ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Problem also with Windows P3 binaries. fwiw, Alan Isaac Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.0.4' >>> A = numpy.array(['a','aa','b']) >>> B = numpy.array(['d','e']) >>> A.searchsorted(B) array([3, 0]) >>> ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
from docstring in multiarraymodule.c
/** @brief Use bisection of sorted array to find first entries >= keys.
*
* For each key use bisection to find the first index i s.t. key <= arr[i].
* When there is no such index i, set i = len(arr). Return the results in
ret.
* All arrays are assumed contiguous on entry and both arr and key must be
of<-
* the same comparable type. <-
*
* @param arr contiguous sorted array to be searched.
* @param key contiguous array of keys.
* @param ret contiguous array of intp for returned indices.
* @return void
*/
static void
local_search_left(PyArrayObject *arr, PyArrayObject *key, PyArrayObject
*ret)
In particular:
* All arrays are assumed contiguous on entry and both arr and key must be
of<-
* the same comparable type. <-
A and B are not of the same type ('|S2' is not '|S1').
This should be mentioned somewhere more accessible.
L.
On 1/31/08, Alan G Isaac <[EMAIL PROTECTED]> wrote:
>
> Problem also with Windows P3 binaries.
> fwiw,
> Alan Isaac
>
> Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310
> 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy
> >>> numpy.__version__
> '1.0.4'
> >>> A = numpy.array(['a','aa','b'])
> >>> B = numpy.array(['d','e'])
> >>> A.searchsorted(B)
> array([3, 0])
> >>>
>
>
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
--
Lorenzo Bolla
[EMAIL PROTECTED]
http://lorenzobolla.emurse.com/
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] matrix <-> ndarray bug
On Thu, 31 Jan 2008, dmitrey apparently wrote: > already fixed in more recent versions? Yes, at least it's fixed in my 1.0.4. By the way, do you know about the ``A`` attribute of matrices? Cheers, Alan Isaac ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] matrix <-> ndarray bug
Alan G Isaac wrote: > On Thu, 31 Jan 2008, dmitrey apparently wrote: > >> already fixed in more recent versions? >> > > Yes, at least it's fixed in my 1.0.4. > > By the way, do you know about the ``A`` attribute of matrices? > Yes, I know, but numpy.ndarray has no the attribute (I had said it's better to be present, as well as .M for matrices, but numpy developers hadn't paid attention), and since I don't know does user pass ndarray or matrix to a func, it's better using asfarray(a) than a.A (later will fail on ndarrays) Regards, D. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
oops. it fails also on an SGI Altix with Suse Linux on it:
Linux pico 2.6.16.27-0.9-default #1 SMP Tue Feb 13 09:35:18 UTC 2007 ia64
ia64 ia64 GNU/Linux
In [33]: A = numpy.array(['a','aa','b'])
In [34]: B = numpy.array(['d','e'])
In [35]: A.searchsorted(B)
Out[35]: array([3, 0])
In [36]: numpy.__version__
Out[36]: '1.0.5.dev4567'
The problem seems to be in the different dtypes of A and B.
Using the same dtype '|S2' it works fine.
In [38]: A = numpy.array(['a','aa','b'])
In [39]: A.dtype
Out[39]:
dtype('|S2')
In [40]: B = numpy.array(['d','e'])
In [41]: A.searchsorted(B)
Out[41]: array([3, 0])
In [42]: B = numpy.array(['d','e'], dtype='|S2')
In [43]: A.searchsorted(B)
Out[43]: array([3, 3])
L.
On 1/31/08, Matthieu Brucher <[EMAIL PROTECTED]> wrote:
>
> No problem for me (also a svn version) :
>
> Python 2.5.1 (r251:54863, Oct 30 2007, 13:54:11)
> [GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2
>
> >>> import numpy
> >>> A = numpy.array(['a','aa','b'])
> >>> B = numpy.array(['d','e'])
> >>> A.searchsorted(B)
> array([3, 3])
>
> Matthieu
>
> 2008/1/31, lorenzo bolla <[EMAIL PROTECTED]>:
> >
> > I use a dev version (1.0.5.dev4567).
> > L.
> >
> >
> > On 1/31/08, James Philbin <[EMAIL PROTECTED]> wrote:
> > >
> > > Hmmm. Just downloaded and installed 1.0.4 and i'm still getting this
> > > error. Are you guys using the bleeding edge version or the official
> > > 1.0.4 tarball from the webpage?
> > >
> > > James
> > > ___
> > > Numpy-discussion mailing list
> > > Numpy-discussion@scipy.org
> > > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> > >
> >
> >
> >
> > --
> > Lorenzo Bolla
> > [EMAIL PROTECTED]
> > http://lorenzobolla.emurse.com/
> >
> > ___
> > Numpy-discussion mailing list
> > Numpy-discussion@scipy.org
> > http://projects.scipy.org/mailman/listinfo/numpy-discussion
> >
> >
>
>
> --
> French PhD student
> Website : http://matthieu-brucher.developpez.com/
> Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn : http://www.linkedin.com/in/matthieubrucher
> ___
> Numpy-discussion mailing list
> Numpy-discussion@scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
--
Lorenzo Bolla
[EMAIL PROTECTED]
http://lorenzobolla.emurse.com/
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Accepted Numpy Core Datatype Formats
Where can I find a list of accepted tokens for the Numpy Core formats as used in numpy.core.rec.fromfile for instance. I'm working with a binary file format that includes single byte buffers and other formats that I need to correctly parse. Any help would be appreciated. Thanks in advance. Mack ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
lorenzo bolla wrote:
> * All arrays are assumed contiguous on entry and both arr and key must
> be of<-
> * the same comparable type. <-
>
> A and B are not of the same type ('|S2' is not '|S1').
> This should be mentioned somewhere more accessible.
It should also raise an exception -- something that *sometimes* works,
and sometimes doesn't is really asking for trouble!
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
[EMAIL PROTECTED]
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Hi, > In particular: > > * All arrays are assumed contiguous on entry and both arr and key must be > of<- > * the same comparable type. <- In which case, this seems to be an overly strict implementation of searchsorted. Surely all that should be required is that the comparison function can take both types. James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
On Jan 31, 2008 9:17 AM, lorenzo bolla <[EMAIL PROTECTED]> wrote: > from docstring in multiarraymodule.c > > /** @brief Use bisection of sorted array to find first entries >= keys. > * > * For each key use bisection to find the first index i s.t. key <= > arr[i]. > * When there is no such index i, set i = len(arr). Return the results in > ret. > * All arrays are assumed contiguous on entry and both arr and key must be > of<- > * the same comparable type. <- > * > * @param arr contiguous sorted array to be searched. > * @param key contiguous array of keys. > * @param ret contiguous array of intp for returned indices. > * @return void > */ > static void > local_search_left(PyArrayObject *arr, PyArrayObject *key, PyArrayObject > *ret) > > In particular: > > * All arrays are assumed contiguous on entry and both arr and key must be > of<- > * the same comparable type. <- > > Heh. I knew there was a reason I documented that subroutine, I had forgotten about that. Anyway, I think an exception should be thrown in the higher level routine that sets up the call when there is a type mismatch. Some typecasting might also be appropriate. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
Well, i've digged around in the source code and here is a patch which
makes it work for the case I wanted:
--- multiarraymodule.c.old 2008-01-31 17:42:32.0 +
+++ multiarraymodule.c 2008-01-31 17:43:43.0 +
@@ -2967,7 +2967,10 @@
char *parr = arr->data;
char *pkey = key->data;
intp *pret = (intp *)ret->data;
-int elsize = arr->descr->elsize;
+
+int elsize1 = arr->descr->elsize;
+int elsize2 = key->descr->elsize;
+
intp i;
for(i = 0; i < nkeys; ++i) {
@@ -2975,14 +2978,14 @@
intp imax = nelts;
while (imin < imax) {
intp imid = imin + ((imax - imin) >> 2);
-if (compare(parr + elsize*imid, pkey, key) < 0)
+if (compare(parr + elsize1*imid, pkey, key) < 0)
imin = imid + 1;
else
imax = imid;
}
*pret = imin;
pret += 1;
-pkey += elsize;
+pkey += elsize2;
}
}
@@ -3008,7 +3011,10 @@
char *parr = arr->data;
char *pkey = key->data;
intp *pret = (intp *)ret->data;
-int elsize = arr->descr->elsize;
+
+int elsize1 = arr->descr->elsize;
+int elsize2 = key->descr->elsize;
+
intp i;
for(i = 0; i < nkeys; ++i) {
@@ -3016,14 +3022,14 @@
intp imax = nelts;
while (imin < imax) {
intp imid = imin + ((imax - imin) >> 2);
-if (compare(parr + elsize*imid, pkey, key) <= 0)
+if (compare(parr + elsize1*imid, pkey, key) <= 0)
imin = imid + 1;
else
imax = imid;
}
*pret = imin;
pret += 1;
-pkey += elsize;
+pkey += elsize2;
}
}
James
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
On Jan 31, 2008 10:33 AM, James Philbin <[EMAIL PROTECTED]> wrote: > Hi, > > > In particular: > > > > * All arrays are assumed contiguous on entry and both arr and key must > be > > of<- > > * the same comparable type. <- > In which case, this seems to be an overly strict implementation of > searchsorted. Surely all that should be required is that the > comparison function can take both types. > True. The problem is knowing when that is the case. The subroutine in question is at the bottom of the heap and don't know nothin'. IIRC, it just sits there and does the comparison by calling through a pointer with char* arguments. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
> True. The problem is knowing when that is the case. The subroutine in > question is at the bottom of the heap and don't know nothin'. IIRC, it just > sits there and does the comparison by calling through a pointer with char* > arguments. What does the comparison function actually look like for the case of dtype='|Sn'? Is there no way of sending the underlying types to the comparison, so it can throw an exception if the two data types are not supported? James ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
On Jan 31, 2008 10:49 AM, James Philbin <[EMAIL PROTECTED]> wrote:
> Well, i've digged around in the source code and here is a patch which
> makes it work for the case I wanted:
>
> --- multiarraymodule.c.old 2008-01-31 17:42:32.0 +
> +++ multiarraymodule.c 2008-01-31 17:43:43.0 +
> @@ -2967,7 +2967,10 @@
> char *parr = arr->data;
> char *pkey = key->data;
> intp *pret = (intp *)ret->data;
> -int elsize = arr->descr->elsize;
> +
> +int elsize1 = arr->descr->elsize;
> +int elsize2 = key->descr->elsize;
> +
> intp i;
>
> for(i = 0; i < nkeys; ++i) {
> @@ -2975,14 +2978,14 @@
> intp imax = nelts;
> while (imin < imax) {
> intp imid = imin + ((imax - imin) >> 2);
> -if (compare(parr + elsize*imid, pkey, key) < 0)
> +if (compare(parr + elsize1*imid, pkey, key) < 0)
> imin = imid + 1;
> else
> imax = imid;
> }
> *pret = imin;
> pret += 1;
> -pkey += elsize;
> +pkey += elsize2;
> }
> }
>
But is that safe? You have changed the stepping to adjust for the element
size, but there is no guarantee that the comparison works.
Chuck
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
On Jan 31, 2008 10:55 AM, James Philbin <[EMAIL PROTECTED]> wrote: > > True. The problem is knowing when that is the case. The subroutine in > > question is at the bottom of the heap and don't know nothin'. IIRC, it > just > > sits there and does the comparison by calling through a pointer with > char* > > arguments. > > What does the comparison function actually look like for the case of > dtype='|Sn'? Is there no way of sending the underlying types to the > comparison, so it can throw an exception if the two data types are not > supported? > There is an upper level routine that parses the keywords and sets things up. There may even be two upper level routines, but I don't recall. The purpose of the two routines you touched was to pull out a small block of code that could be very simple because of its assumptions. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
James Philbin wrote: > I can't fathom where the comparison functions exist in the code. It > seems that the comparison signature is of the form (void*, void*, > PyArrayObject*), so it doesn't seem possible at the moment to specify > a compare function which can reason about the underlying types of the > two void*'s. However, I think arrays of strings are a common enough > use case that they should work as expected - would it be possible to > extend the comparison type to accept two integers specifying the types > of the arguments? > The problem is due to the use of an older API in type conversion. I think I can provide a fix in a few minutes. The compare function is type-specific and works for strings but requires the same length string for each argument. It is defined as part of the PyArray_Descr object (defined in arraytypes.inc.src). It may be possible to not require contiguous arrays, but that is a separate issue. -Travis O. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
I can't fathom where the comparison functions exist in the code. It seems that the comparison signature is of the form (void*, void*, PyArrayObject*), so it doesn't seem possible at the moment to specify a compare function which can reason about the underlying types of the two void*'s. However, I think arrays of strings are a common enough use case that they should work as expected - would it be possible to extend the comparison type to accept two integers specifying the types of the arguments? James On Jan 31, 2008 6:02 PM, Charles R Harris <[EMAIL PROTECTED]> wrote: > > > > > On Jan 31, 2008 10:55 AM, James Philbin <[EMAIL PROTECTED]> wrote: > > > > > True. The problem is knowing when that is the case. The subroutine in > > > question is at the bottom of the heap and don't know nothin'. IIRC, it > just > > > sits there and does the comparison by calling through a pointer with > char* > > > arguments. > > > > What does the comparison function actually look like for the case of > > dtype='|Sn'? Is there no way of sending the underlying types to the > > comparison, so it can throw an exception if the two data types are not > > supported? > > > > > > > > > > There is an upper level routine that parses the keywords and sets things up. > There may even be two upper level routines, but I don't recall. The purpose > of the two routines you touched was to pull out a small block of code that > could be very simple because of its assumptions. > > > Chuck > > > > ___ > Numpy-discussion mailing list > Numpy-discussion@scipy.org > http://projects.scipy.org/mailman/listinfo/numpy-discussion > > ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] searchsorted bug
James Philbin wrote: > I can't fathom where the comparison functions exist in the code. It > seems that the comparison signature is of the form (void*, void*, > PyArrayObject*), so it doesn't seem possible at the moment to specify > a compare function which can reason about the underlying types of the > two void*'s. However, I think arrays of strings are a common enough > use case that they should work as expected - would it be possible to > extend the comparison type to accept two integers specifying the types > of the arguments? > Try out latest SVN. It should have this problem fixed. Look at the diff to see the fix (note that copies are made if the types are not "exactly" the same --- meaning the same length if variable types). -Travis O. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
