Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Pauli : Thanks for your reply. Using 'wb' instead of 'w' for the file mode evidently makes this problem go away. The mystery is that why does this work with plain lists *and* 'w' and not with numpy arrays and 'w'. In other words, why is it that numpy.array needs the binary mode while list does not? Also, for my own edification, how did you know that my pickle files produced on Windows were contaminated by \r\n line feeds. Thanks again! On Thu, Nov 5, 2009 at 5:47 AM, Pauli Virtanen pav...@iki.fi wrote: Wed, 04 Nov 2009 19:21:57 -0800, Reckoner wrote: Bruce : The file in question was created as shown in the prior e-mail. Here it is again: cPickle.dump(a,open('from32bitxp.pkl','w')) Your pickle files produced on Windows are contaminated by \r\n line feeds. AFAIK, pickle is supposed to be a binary stream. You need to open the files in binary 'wb' mode, not in text 'w' mode. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Thu, Nov 5, 2009 at 14:11, Reckoner recko...@gmail.com wrote: Pauli : Thanks for your reply. Using 'wb' instead of 'w' for the file mode evidently makes this problem go away. The mystery is that why does this work with plain lists *and* 'w' and not with numpy arrays and 'w'. In other words, why is it that numpy.array needs the binary mode while list does not? I'm not sure what to say except because it does. Pickles don't necessarily contain binary data, particularly using the default protocol=0 and only builtin types, but they may. Always use 'wb'. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Thu, 05 Nov 2009 12:11:32 -0800, Reckoner wrote: Pauli : Thanks for your reply. Using 'wb' instead of 'w' for the file mode evidently makes this problem go away. The mystery is that why does this work with plain lists *and* 'w' and not with numpy arrays and 'w'. In other words, why is it that numpy.array needs the binary mode while list does not? Requiring binary is not specific to numpy, but just probably the fact that when you have \r\n linefeeds, pickle ends up trying to load a module called numpy.core.multiarray\r which does not exists. Same thing happens with any other types -- it's just that lists are builtin types and don't need any modules loaded to unpickle properly. Also, for my own edification, how did you know that my pickle files produced on Windows were contaminated by \r\n line feeds. I looked in the two files you attached on Numpy trac, and noticed that they were different (sha1sum). -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Here's an example: On winxp 64-bit: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy import cPickle a = numpy.eye(10) cPickle.dump(a,open('from32bitxp.pkl','w')) import numpy.core.multiarray numpy.__version__ '1.0.4' On linux 64 bit: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import numpy import cPickle cPickle.load(open('from32bitxp.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray numpy.__version__ '1.2.1' import numpy.core.multiarray Note that I transfer the from32bitxp file from the winxp32 machine to the linux host. Also, I've tried this with version 1.3 on winxp and get the same problem on the linux host. Here's more interesting info: On linux: a = numpy.eye(10) cPickle.dump(a,open('from64bitLinux.pkl','w')) upon transferring the file to winxp 32 and on winxp32: cPickle.load(open('from64bitLinux.pkl')) See? No problem going from linux to winxp32; but problems going the other way. Please let me know if you need more info on this. Any help appreciated. On Tue, Nov 3, 2009 at 4:55 AM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 6:31 PM, Reckoner recko...@gmail.com wrote: thanks for the suggestion! I will look into it. The other thing is that the numpy arrays in question are actually embedded in another object. When I convert the numpy arrays into plain lists, and then cPickle them, there is no problem with any of the larger objects. That is the way we are currently working around this issue. Thanks again. On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ Can you provide you provide a small self-contained example of the problem including object creation especially as your example does not import numpy? Really you have to start at the beginning (like pickling and transferring numpy arrays) and then increase the complexity to include the object. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Wed, Nov 4, 2009 at 7:06 AM, Reckoner recko...@gmail.com wrote: Here's an example: On winxp 64-bit: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy import cPickle a = numpy.eye(10) cPickle.dump(a,open('from32bitxp.pkl','w')) import numpy.core.multiarray numpy.__version__ '1.0.4' On linux 64 bit: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import numpy import cPickle cPickle.load(open('from32bitxp.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray numpy.__version__ '1.2.1' import numpy.core.multiarray I wonder if this is a numpy version problem. Do you have a windows machine with a more recent version of numpy on it? snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Thanks for your reply. No. I just tried it with the latest Windows XP 32-bit version Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy numpy.__version__ '1.3.0' Same result on the Linux side: ImportError: No module named multiarray. Thanks! On Wed, Nov 4, 2009 at 7:17 AM, Charles R Harris charlesr.har...@gmail.com wrote: On Wed, Nov 4, 2009 at 7:06 AM, Reckoner recko...@gmail.com wrote: Here's an example: On winxp 64-bit: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy import cPickle a = numpy.eye(10) cPickle.dump(a,open('from32bitxp.pkl','w')) import numpy.core.multiarray numpy.__version__ '1.0.4' On linux 64 bit: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import numpy import cPickle cPickle.load(open('from32bitxp.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray numpy.__version__ '1.2.1' import numpy.core.multiarray I wonder if this is a numpy version problem. Do you have a windows machine with a more recent version of numpy on it? snip Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Wed, Nov 4, 2009 at 8:06 AM, Reckoner recko...@gmail.com wrote: Here's an example: On winxp 64-bit: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy import cPickle a = numpy.eye(10) cPickle.dump(a,open('from32bitxp.pkl','w')) import numpy.core.multiarray numpy.__version__ '1.0.4' On linux 64 bit: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import numpy import cPickle cPickle.load(open('from32bitxp.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray numpy.__version__ '1.2.1' import numpy.core.multiarray Note that I transfer the from32bitxp file from the winxp32 machine to the linux host. Also, I've tried this with version 1.3 on winxp and get the same problem on the linux host. Here's more interesting info: On linux: a = numpy.eye(10) cPickle.dump(a,open('from64bitLinux.pkl','w')) upon transferring the file to winxp 32 and on winxp32: cPickle.load(open('from64bitLinux.pkl')) See? No problem going from linux to winxp32; but problems going the other way. Please let me know if you need more info on this. Any help appreciated. On Tue, Nov 3, 2009 at 4:55 AM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 6:31 PM, Reckoner recko...@gmail.com wrote: thanks for the suggestion! I will look into it. The other thing is that the numpy arrays in question are actually embedded in another object. When I convert the numpy arrays into plain lists, and then cPickle them, there is no problem with any of the larger objects. That is the way we are currently working around this issue. Thanks again. On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ Can you provide you provide a small self-contained example of the problem including object creation especially as your example does not import numpy? Really you have to start at the beginning (like pickling and transferring numpy arrays) and then increase the complexity to include the object. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Hi, I did not see the file 'from32bitxp.pkl'. It would really help if you can provide the full example that includes the creation of the complete object so at least people could try doing the same process with the object that you
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Bruce : The file in question was created as shown in the prior e-mail. Here it is again: cPickle.dump(a,open('from32bitxp.pkl','w')) Thanks! On Wed, Nov 4, 2009 at 3:56 PM, Bruce Southey bsout...@gmail.com wrote: On Wed, Nov 4, 2009 at 8:06 AM, Reckoner recko...@gmail.com wrote: Here's an example: On winxp 64-bit: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy import cPickle a = numpy.eye(10) cPickle.dump(a,open('from32bitxp.pkl','w')) import numpy.core.multiarray numpy.__version__ '1.0.4' On linux 64 bit: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import numpy import cPickle cPickle.load(open('from32bitxp.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray numpy.__version__ '1.2.1' import numpy.core.multiarray Note that I transfer the from32bitxp file from the winxp32 machine to the linux host. Also, I've tried this with version 1.3 on winxp and get the same problem on the linux host. Here's more interesting info: On linux: a = numpy.eye(10) cPickle.dump(a,open('from64bitLinux.pkl','w')) upon transferring the file to winxp 32 and on winxp32: cPickle.load(open('from64bitLinux.pkl')) See? No problem going from linux to winxp32; but problems going the other way. Please let me know if you need more info on this. Any help appreciated. On Tue, Nov 3, 2009 at 4:55 AM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 6:31 PM, Reckoner recko...@gmail.com wrote: thanks for the suggestion! I will look into it. The other thing is that the numpy arrays in question are actually embedded in another object. When I convert the numpy arrays into plain lists, and then cPickle them, there is no problem with any of the larger objects. That is the way we are currently working around this issue. Thanks again. On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ Can you provide you provide a small self-contained example of the problem including object creation especially as your example does not import numpy? Really you have to start at the beginning (like pickling and transferring numpy arrays) and then increase the complexity to include the object. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Hi, I
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
FYI, I uploaded the two files in question to the numpy ticket http://projects.scipy.org/numpy/ticket/1284 Thanks! On Wed, Nov 4, 2009 at 3:56 PM, Bruce Southey bsout...@gmail.com wrote: On Wed, Nov 4, 2009 at 8:06 AM, Reckoner recko...@gmail.com wrote: Here's an example: On winxp 64-bit: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type help, copyright, credits or license for more information. import numpy import cPickle a = numpy.eye(10) cPickle.dump(a,open('from32bitxp.pkl','w')) import numpy.core.multiarray numpy.__version__ '1.0.4' On linux 64 bit: Python 2.5.4 (r254:67916, Feb 5 2009, 19:52:35) [GCC 4.1.2 20071124 (Red Hat 4.1.2-42)] on linux2 Type help, copyright, credits or license for more information. import numpy import cPickle cPickle.load(open('from32bitxp.pkl')) Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named multiarray numpy.__version__ '1.2.1' import numpy.core.multiarray Note that I transfer the from32bitxp file from the winxp32 machine to the linux host. Also, I've tried this with version 1.3 on winxp and get the same problem on the linux host. Here's more interesting info: On linux: a = numpy.eye(10) cPickle.dump(a,open('from64bitLinux.pkl','w')) upon transferring the file to winxp 32 and on winxp32: cPickle.load(open('from64bitLinux.pkl')) See? No problem going from linux to winxp32; but problems going the other way. Please let me know if you need more info on this. Any help appreciated. On Tue, Nov 3, 2009 at 4:55 AM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 6:31 PM, Reckoner recko...@gmail.com wrote: thanks for the suggestion! I will look into it. The other thing is that the numpy arrays in question are actually embedded in another object. When I convert the numpy arrays into plain lists, and then cPickle them, there is no problem with any of the larger objects. That is the way we are currently working around this issue. Thanks again. On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ Can you provide you provide a small self-contained example of the problem including object creation especially as your example does not import numpy? Really you have to start at the beginning (like pickling and transferring numpy arrays) and then increase the complexity to include the object. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Hi, I did not see the file
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Mon, Nov 2, 2009 at 6:31 PM, Reckoner recko...@gmail.com wrote: thanks for the suggestion! I will look into it. The other thing is that the numpy arrays in question are actually embedded in another object. When I convert the numpy arrays into plain lists, and then cPickle them, there is no problem with any of the larger objects. That is the way we are currently working around this issue. Thanks again. On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ Can you provide you provide a small self-contained example of the problem including object creation especially as your example does not import numpy? Really you have to start at the beginning (like pickling and transferring numpy arrays) and then increase the complexity to include the object. Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Mon, Nov 2, 2009 at 15:42, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Nope! Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Create a new ticket on the Trac: http://projects.scipy.org/numpy/ If you can attach the offending pickle files, that would be ideal. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
thanks for the suggestion! I will look into it. The other thing is that the numpy arrays in question are actually embedded in another object. When I convert the numpy arrays into plain lists, and then cPickle them, there is no problem with any of the larger objects. That is the way we are currently working around this issue. Thanks again. On Mon, Nov 2, 2009 at 2:43 PM, Bruce Southey bsout...@gmail.com wrote: On Mon, Nov 2, 2009 at 2:42 PM, Reckoner recko...@gmail.com wrote: Anybody have any ideas here? Otherwise, I'm thinking this should be posted to the numpy bugs list. What's the best way to report a bug of this kind? Thanks! On Fri, Oct 30, 2009 at 5:48 PM, Reckoner recko...@gmail.com wrote: Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Have you have tried the other Cookbook approaches: http://www.scipy.org/Cookbook/InputOutput Like using numpy's own array io functions - load/save(z)? (seems to work between 64-bit Windows 7 and 64-bit Linux - each has different numpy versions) Bruce ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
On Fri, Oct 30, 2009 at 13:13, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. You can import numpy.core.multiarray on both machines? -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] persistent ImportError: No module named multiarray when moving cPickle files between machines
Robert Kern wrote: You can import numpy.core.multiarray on both machines? Yes. For each machine separately, you can cPickle files with numpy arrays without problems loading/dumping. The problem comes from transferring the win32 cPickle'd files to Linux 64 bit and then trying to load them. Transferring cPickle'd files that do *not* have numpy arrays work as expected. In other words, cPICKLE'd lists transfer fine back and forth between the two machines. In fact, we currently get around this problem by converting the numpy arrays to lists, transferring them, and then re-numpy-ing them on the respective hosts thanks. On Fri, Oct 30, 2009 at 11:13 AM, Reckoner recko...@gmail.com wrote: Hi, % python -c 'import numpy.core.multiarray' works just fine, but when I try to load a file that I have transferred from another machine running Windows to one running Linux, I get: % python -c 'import cPickle;a=cPickle.load(open(matrices.pkl))' Traceback (most recent call last): File string, line 1, in module ImportError: No module named multiarray otherwise, cPickle works normally when transferring files that *do* not contain numpy arrays. I am using version 1.2 on both machines. It's not so easy for me to change versions, by the way, since this is the version that my working group has decided on to standardize on for this effort. Any help appreciated. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion