[Numpy-discussion] py2/py3 pickling

2015-08-23 Thread Chris Laumann 
Hi all-

Is there documentation about the limits and workarounds for py2/py3 
pickle/np.save/load compatibility? I haven't found anything except developer 
bug tracking discussions (eg. #4879 in github numpy).

The kinds of errors you get can be really obscure when save/loading complicated 
objects or pickles containing numpy scalars. It's really unclear to me why the 
following shouldn't work -- it doesn't have anything apparent to do with string 
handling and unicode.

Run in py2:

import pickle
import numpy as np

a = np.float64(0.99)
pickle.dump(a, open('test.pkl', 'wb'))

And then in py3:

import pickle
import numpy as np

b = pickle.load(open('test.pkl', 'rb'))

And you get:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xae in position 0: ordinal 
not in range(128)

If you force encoding='bytes' in the load, it works.

Is this explained anywhere?

Best, C
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy helper function for __getitem__?

2015-08-23 Thread Fabien 
On 08/23/2015 08:08 PM, Stephan Hoyer wrote:
> I don't think NumPy has a function like this (at least, not exposed to
> Python), but I wrote one for xray, "expanded_indexer", that you are
> welcome to borrow:

Hi Stephan,

that's perfect, thanks!

Fabien

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] Numpy helper function for __getitem__?

2015-08-23 Thread Stephan Hoyer 
I don't think NumPy has a function like this (at least, not exposed to Python), 
but I wrote one for xray, "expanded_indexer", that you are welcome to borrow:
https://github.com/xray/xray/blob/v0.6.0/xray/core/indexing.py#L10










​Stephan



On Sunday, Aug 23, 2015 at 7:54 PM, Fabien , wrote:
Folks,


My search engine was not able to help me on this one, possibly because I 

don't know exactly *what* I am looking for.


I need to override __getitem__ for a class that wrapps a numpy array. I 

know the dimensions of my array (which can be variable from instance to 

instance), and I know what I want to do: for one preselected dimension, 

I need to select another slice than requested by the user, do something 

with the data, and return the variable.


I am looking for a function that helps me to "clean" the input of 

__getitem__. There are so many possible cases, when the user uses [:] or 

[..., 1:2] or [0, ..., :] and so forth. But all these cases have an 

equivalent index array of len(ndimensions) with only valid slice() 

objects in it. This array would be much easier for me to work with.


in pseudo code:


def __getitem__(self, item):

 # clean input

 item = np.clean_item(item, ndimensions=4)

 # Ok now item is guaranteed to be of len 4

 item[2] = slice()

 # Continue

 etc.


Is there such a function in numpy?


I hope I have been clear enough... Thanks a lot!


Fabien


___

NumPy-Discussion mailing list

NumPy-Discussion@scipy.org

http://mail.scipy.org/mailman/listinfo/numpy-discussion___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

[Numpy-discussion] Numpy helper function for __getitem__?

2015-08-23 Thread Fabien 
Folks,

My search engine was not able to help me on this one, possibly because I 
don't know exactly *what* I am looking for.

I need to override __getitem__ for a class that wrapps a numpy array. I 
know the dimensions of my array (which can be variable from instance to 
instance), and I know what I want to do: for one preselected dimension, 
I need to select another slice than requested by the user, do something 
with the data, and return the variable.

I am looking for a function that helps me to "clean" the input of 
__getitem__. There are so many possible cases, when the user uses [:] or 
[..., 1:2] or [0, ..., :] and so forth. But all these cases have an 
equivalent index array of len(ndimensions) with only valid slice() 
objects in it. This array would be much easier for me to work with.

in pseudo code:

def __getitem__(self, item):
 # clean input
 item = np.clean_item(item, ndimensions=4)
 # Ok now item is guaranteed to be of len 4
 item[2] = slice()
 # Continue
 etc.

Is there such a function in numpy?

I hope I have been clear enough... Thanks a lot!

Fabien

___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion