[Numpy-discussion] str/bytes object from arr.data?
Hi, Anybody knows a way to get a str object (bytes for Python >= 2.6) out of a buffer object (i.e. the .data attribute of ndarrays) without copying data? I need this for avoid creating a new function that deals with buffer objects (instead of reusing the one for str/byte objects that I already have). So, a matter of laziness :-) Thanks, -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] str/bytes object from arr.data?
As str objects are supposed to be immutable, I think anything "official" that makes a string from a numpy array is supposed to copy the data. But I think you can use ctypes to wrap a pointer and a length as a python string. Zach On Sep 27, 2010, at 8:28 AM, Francesc Alted wrote: > Hi, > > Anybody knows a way to get a str object (bytes for Python >= 2.6) > out of > a buffer object (i.e. the .data attribute of ndarrays) without copying > data? > > I need this for avoid creating a new function that deals with buffer > objects (instead of reusing the one for str/byte objects that I > already > have). So, a matter of laziness :-) > > Thanks, > > -- > Francesc Alted > ___ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Matlab IO Warning in mio5.py
On Mon, Sep 27, 2010 at 12:07 PM, Charles R Harris < charlesr.har...@gmail.com> wrote: > > On Sun, Sep 26, 2010 at 11:03 PM, Ben Longstaff >> wrote: >> >> >>> > Warning Message >> >>> > C:\Python26\lib\site-packages\scipy\io\matlab\mio5.py:90: >> >>> > RuntimeWarning: >> >>> > __builtin__.file size changed, may indicate binary incompatibility >> >>> > from mio5_utils import VarReader5 >> >>> > >> >>> > Hello, World! >> >>> >> >>> Which numpy and scipy versions are you using? e.g. scipy.__version__ >> >>> np.__version__ >> >>> >> >>> This looks like a warning from cython, but the versions are still >> >>> binary compatible, and there shouldn't be any problems. >> >>> (with all numpy except 1.4.0 and 2.x in repository) >> >>> >> > > Didn't Ralph catch the warnings on import for the released binaries? Maybe > he missed one. Also, I don't recall seeing seeing that one before, not that > it is necessarily a problem. > Hmm, missed this one, also can't reproduce it on either XP or OS X with the same numpy/scipy versions. If these warnings are platform-dependent shouldn't that be considered a Cython bug? Related commits are r6331 and r6512 in scipy. Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Asking for opinions: Priops
2010/9/23 Dag Sverre Seljebotn : > Essentially, perhaps what you have sketched up + an ability to extend > the graph with object conversion routes would be perfect for my own > uses. So you can define a function with overloads (A, B) and (A, C), but > also that objects of type D can be converted to C (and how). For > instance, consider: > > np.array([1,2,3]) + [1,2,3] > > Here, list-> array could be handled through a defined coercion to array, > rather than having to add an overload for list for every method taking > an array. This seems to be a good thing. Let's reason about this for some moment. Atm, the relation is *not* transitive. (I.e., (A, B) and (B, C) does imply nothing for (A, C). A, B, C classes.) But such kind of transitivity is what you mean, if (A, B) is defined and there is a graph egde in the "conversion graph" (B, C), then (A, C) can choose the (A, B) route via C -> B conversion; here the notation (B, C) in the conversion graph means "conversion from C to B". I don't see a clear solution at the end satisfying me. It seems that one really has to conduct a search in the additional conversion graph. Since this is expensive, I believe it would probably be good to derive a "ConversionPriop" from priop.Priop. What are your thoughts now? I feel it useful to add this conversion graph, since it creates many egdes in the resulting effective coercion graph, which do not have all to be specified explicitly, but which exist. Before, this conversion graph exists already in the sense of subclassing, i.e., if your second object is a subclass of `list`, in your example, it will be sufficient to define the edge with `list`, and it will find the edge. Would this suffice for your needs? It could be that this is even safer than magic conversion pathes. Maybe it's even better, more straightforward. I'm really unsure about this. Please give me your feedback. > np.array([1,2,3]) + [1,2,3] For this, a) numpy.ndarray could be subclassing list, if it doesn't do already, but then it would look like list concatenation. b) It would be sufficient to add the edge with `list` as outlined before. So, now I tend to *not* adding the functionality ... :-/ Maybe the strongest argument in favour of adding it nevertheless is, that, in our example, ndarray can be seen at a "view" of a list. So operations with objects that can be seen as something else should be supported, especially if the class it is seen as is more deep in the class hierarchy. This means, edges (ndarray, ndarray) in fact should match on (ndarray, list), although `ndarray` is more deep than `list`. (Opposed to the case one defines (ndarray, list), and this matches (ndarrary, list_subclass), seeing only the `list` functionality in `list_subclass`.) Would it, having this in mind, suffice to add a transitivity not only for subclasses, but also for superclasses? If (A, B) shall be coerced and C is a subclass of B, i.e. B a superclass of C, then (A, B) should translate to (A, C), using C(b) for the operand? This means, C *extends* the functionality of B only. I'm feeling quite good with this, because it's less arbitrary than allowing for *all* conversion routes. Actually it's nothing more than giving the class C as the conversion function, since classes are callable. And since not all subclasses maybe need more than a superclass instance for construction ..., the graph still has to created manually. Looking further into this, it seems to suffice classes which can be constructed from their superclass instances only. E.g. class C just needs to be registered as an "extending class", and if in (A, B) it holds that ``isinstance(b, C)`` while (A, B) is not defined but (A, C), then the (A, C) edge will be called via the C(b) constructor. Conversion would just happen using a *take* argument to the registration, optionally being a string defining the kwarg used in the C constructor for construction from base class instances. E.g.: priop.extender(numpy.ndarray, constructor=numpy.asarray, extends=list) priop.extender(my_class, take='universal') # *universal* may be some late kwarg in the constructor. # Automatically applies to all classes `my_class` derives from. Unfortunately, ndarray isn't a subclass of list. Maybe this would be worth a NEP, if it is technically possible. Otherwise, the first form would suffice it. ? It got a bit longish, Friedrich P.S.: I like the fact that I understood that "conversion a->b" means "B extends A". As opposed to the fact "B functionality is a subset of A functionality". ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Asking for opinions: Priops
Friedrich Romstedt wrote: > 2010/9/23 Dag Sverre Seljebotn : > >> Essentially, perhaps what you have sketched up + an ability to extend >> the graph with object conversion routes would be perfect for my own >> uses. So you can define a function with overloads (A, B) and (A, C), but >> also that objects of type D can be converted to C (and how). For >> instance, consider: >> >> np.array([1,2,3]) + [1,2,3] >> >> Here, list-> array could be handled through a defined coercion to array, >> rather than having to add an overload for list for every method taking >> an array. >> > > This seems to be a good thing. Let's reason about this for some > moment. Atm, the relation is *not* transitive. (I.e., (A, B) and (B, > C) does imply nothing for (A, C). A, B, C classes.) But such kind of > transitivity is what you mean, if (A, B) is defined and there is a > graph egde in the "conversion graph" (B, C), then (A, C) can choose > the (A, B) route via C -> B conversion; here the notation (B, C) in > the conversion graph means "conversion from C to B". > > I don't see a clear solution at the end satisfying me. It seems that > one really has to conduct a search in the additional conversion graph. > Since this is expensive, I believe it would probably be good to > derive a "ConversionPriop" from priop.Priop. > > What are your thoughts now? > You can just cache all lookups in the conversion graph, so that after some initialization all lookups are O(1). There's a limited number of types in a Python runtime, and the actual distinct lookups performed are likely to not be many. I don't see that as a problem at all. I'll try to remember to get back to the rest in a week or so...I'm handing in my MSc on Thursday :-) Dag Sverre ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] str/bytes object from arr.data?
A Monday 27 September 2010 15:04:14 Zachary Pincus escrigué: > As str objects are supposed to be immutable, I think anything > "official" that makes a string from a numpy array is supposed to copy > the data. But I think you can use ctypes to wrap a pointer and a > length as a python string. Yeah, ctypes is very powerful indeed. Thanks! -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] mean of empty sequence gives nan
In numpy 1.5.0, I got the following for mean of an empty sequence (or array): In [21]: mean([]) Warning: invalid value encountered in double_scalars Out[21]: nan Is this behaviour expected ? Also, would it be possible to have more explicit warning messages about the problem being related numpy ? It seems these message cause quite a bit of confusion to users. The following would be more helpful : Warning: invalid value encountered in numpy double_scalars ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] mean of empty sequence gives nan
On Mon, Sep 27, 2010 at 17:51, Peter Butterworth wrote: > In numpy 1.5.0, I got the following for mean of an empty sequence (or array): > > In [21]: mean([]) > Warning: invalid value encountered in double_scalars > Out[21]: nan > > Is this behaviour expected ? np.sum([]) / len([]) -> 0.0 / 0 -> nan, so yes. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
