Alexandre Vassalotti alexan...@peadrop.com added the comment:
Committed fix for 3.x in r74013 and for 2.x in r74014.
--
resolution: - fixed
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2389
Alexandre Vassalotti alexan...@peadrop.com added the comment:
I know believe that arrays should be pickled as a list of values on
Python 2.x. Doing otherwise makes it impossible to unpickle arrays
coming from Python 2.x using Python 3.x, since pickle on Python 3
decodes all the strings from 2.x
Alexandre Vassalotti alexan...@peadrop.com added the comment:
I would like to commit my patch later this week. So if you see any issue
with the patch, please speak up.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2389
Changes by Alexandre Vassalotti alexan...@peadrop.com:
Removed file: http://bugs.python.org/file11096/fix_array_pickling.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2389
___
Alexandre Vassalotti alexan...@peadrop.com added the comment:
Here's a patch that implements the solution I described in msg85298.
Please give it a good review:
http://codereview.appspot.com/87072
--
Added file: http://bugs.python.org/file14369/portable_array_pickling.diff
Alexandre Vassalotti alexan...@peadrop.com added the comment:
Ah, I just remembered the smart way I had devised some time ago to
handle this issue without changing the constructor of array.array. The
trick would be to add a __reduce__ method to array.array. This method
would return a special
Changes by Collin Winter coll...@gmail.com:
--
nosy: +collinwinter
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue2389
___
___
Python-bugs-list
Alexandre Vassalotti [EMAIL PROTECTED] added the comment:
I'm all in for a standardized representation of array's pickles (with
width and endianness preserved). However to happen, we will either need
to change array's constructor to support at least the byte-order
specification (like struct) or
Martin v. Löwis [EMAIL PROTECTED] added the comment:
I think changing the array constructor is fairly easy: just pick a set
of codes that are defined to be platform-neutral (i.e. for each size two
codes, one for each endianness). For example, the control characters
(\0..\x1F) could be used in
Hrvoje Nikšić [EMAIL PROTECTED] added the comment:
Unfortunately dumping the internal representation of non-long arrays
won't work, for several reasons. First, it breaks when porting pickles
between platforms of different endianness such as Intel and SPARC.
Then, it ignores the considerable
Martin v. Löwis [EMAIL PROTECTED] added the comment:
I like to challenge the view what correct behavior is here. If I
pickle an array of 32-bit integer values on one system, and unpickle it
as an array of 64-bit integer values on a different system, is that
correct, or incorrect?
IMO, correct
Hrvoje Nikšić [EMAIL PROTECTED] added the comment:
I think preserving integer width is a good idea because it saves us from
having to throw overflow errors when unpickling to machines with
different width of C types. The cost is that pickling/unpickling the
array might change the array's
Guido van Rossum [EMAIL PROTECTED] added the comment:
Instead of sticking to network byte order, I propose to include byte
order information in the pickle (for example as '' or '' like struct
does), so that pickling/unpickling between the same-endianness
architectures doesn't have to convert
Guido van Rossum [EMAIL PROTECTED] added the comment:
Wouldn't that be lots and lots slower? I believe speed is one of the
reasons why the binary representation is currently dumped.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2389
Alexandre Vassalotti [EMAIL PROTECTED] added the comment:
The slowdown depends of the array type. The patch makes array unpickling
a few orders of magnitude slower (i.e. between 4 and 15 times slower
depending of the array type). In general, pickling is about as fast as
with the binary
Alexandre Vassalotti [EMAIL PROTECTED] added the comment:
I don't see why this cannot be fixed easily. All we need to do is fix
the __reduce__ method of array objects to emit a list--i.e. with
array.tolist()--instead of a memory string. Since the reduce protocol is
just a fancy way to store the
Hrvoje Nikšić [EMAIL PROTECTED] added the comment:
I guess it went unnoticed due to prevalence of little-endian 32-bit
machines. With 64-bit architectures becoming more and more popular,
this might become a bigger issue.
Raymond, why do you think fixing this bug would complicate porting to
Guido van Rossum [EMAIL PROTECTED] added the comment:
Agreed, this has been broken for a long time, and few people have
noticed or complained. Let's wait.
--
assignee: gvanrossum -
versions: +Python 2.7, Python 3.1 -Python 2.6, Python 3.0
___
Raymond Hettinger [EMAIL PROTECTED] added the comment:
At this point, I think it better to wait until Py2.7/3.1. Changing it
now would just complicate efforts to port from 2.5 to 2.6 to 3.0.
Guido, do you agree?
--
assignee: - gvanrossum
nosy: +rhettinger
Benjamin Peterson [EMAIL PROTECTED] added the comment:
Ping.
--
nosy: +benjamin.peterson
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2389
___
___
Changes by Jesús Cea Avión [EMAIL PROTECTED]:
--
nosy: +jcea
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2389
__
___
Python-bugs-list mailing list
Unsubscribe:
Guido van Rossum [EMAIL PROTECTED] added the comment:
This looks indeed wrong. Unfortunately it also looks hard to fix in a
way that won't break unpickling arrays pickled by a previous Python
version. We won't be able to fix this in 2.5 (it'll be a new feature)
but we should try to fix this in
Hrvoje Nikšić [EMAIL PROTECTED] added the comment:
Here is an example that directly demonstrates the bug. Pickling on x86_64:
Python 2.5.1 (r251:54863, Mar 21 2008, 13:06:31)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type help, copyright, credits or license for more
New submission from Hrvoje Nikšić [EMAIL PROTECTED]:
It would seem that pickling arrays directly exposes the underlying
machine words, making the pickle non-portable to platforms with
different layout of array elements. The guts of array.__reduce__ look
like this:
if (array-ob_size 0)
24 matches
Mail list logo
