[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Alexandre Vassalotti added the comment: I fixed up the last few review comments and submitted the patch. Thank you for the help! -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Roundup Robot added the comment: New changeset bd71352e950f by Alexandre Vassalotti in branch 'default': Issue #6784: Strings from Python 2 can now be unpickled as bytes objects. http://hg.python.org/cpython/rev/bd71352e950f -- nosy: +python-dev ___ P

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Serhiy Storchaka added the comment: And what about an issue mentioned in msg153659? -- nosy: +serhiy.storchaka ___ Python tracker ___ _

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Antoine Pitrou added the comment: How about updating the documentation as well? -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Alexandre Vassalotti added the comment: I cleaned up the patch. I will submit it tonight if there is no major objections. -- Added file: http://bugs.python.org/file33019/pickle_python2_str_as_bytes.diff ___ Python tracker

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: I have fixed most of the nits in this patch, except for: 1) the intermediate bytes object being created; inlining is an option, as storchaka suggested, but I'd rather have you decide what it should become before implementing it; 2) make clinic gives me ./

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: Hi Alexandre, Attached is a diff based on r87793:0c508d87f80b. Merlijn -- Added file: http://bugs.python.org/file33011/bytestrpickle.diff ___ Python tracker

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24907/pickle_bytes_tests.diff ___ Python tracker ___ ___ Python-bugs-lis

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24568/pickle.py.patch ___ Python tracker ___ ___ Python-bugs-list mailin

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24906/pickle_bytes_code.diff ___ Python tracker ___ ___ Python-bugs-list

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24719/pickle_bytestr.patch ___ Python tracker ___ ___ Python-bugs-list m

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24688/test_pickle.diff ___ Python tracker ___ ___ Python-bugs-list maili

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24640/BytestrPickler_c.diff ___ Python tracker ___ ___ Python-bugs-list

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Alexandre Vassalotti added the comment: Could you provide a single patch with the implementation and the tests together? I will try to find some time this week to review this. -- assignee: docs@python -> alexandre.vassalotti priority: normal -> high stage: -> patch review versions: +Py

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Florent Xicluna : -- nosy: +flox ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Mikhail Korobov : -- nosy: +kmike ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: ...and the tests to go with that. -- Added file: http://bugs.python.org/file24907/pickle_bytes_tests.diff ___ Python tracker ___ _

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: Based on the discussion on python-dev [1], this is an updated implementation that uses encoding='bytes' to signal str->bytes behaviour. http://mail.python.org/pipermail/python-dev/2012-March/117536.html -- Added file: http://bugs.python.org/file2490

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24714/pickle_bytestr.patch ___ Python tracker ___ ___ Python-bugs-list m

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: OK, and now a version that's not broken... I forgot to initialize self->bytestr for PicklerObject/UnpicklerObject. *puts on the you-broke-the-build-hat* Except for test_packaging.test_caches, this version passes all tests -- test_packaging.test_caches, whic

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Removed file: http://bugs.python.org/file24567/test_bytestrpickle.py ___ Python tracker ___ ___ Python-bugs-list

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: And a complete patch that implements the tests, the python implementation and the C implementation. I'm not completely happy with the code duplication in read_string/read_binstring/read_short_binstring C implementation, so that might be an improvement (howe

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: Added tests in Lib/test format. After applying pickle.py.patch and BytestrPickler_c.diff, ./python -m test -v -m PyPicklerBytestrTests test_pickle returns 12 tests, no errors, while ./python -m test -v -m CPicklerBytestrTests test_pickle only passes

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: Ok, this is my first attempt at the Pickler part of the C implementation. I'll have to adapt the python implementation to match this one. All BytestrPicklerTests in test_bytestrpickle.py pass, and ./python -m test -G -v test_pickle passes. Comments on styl

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: P.S. (sorry for forgetting this in the original post ;-)) Both ./python -m test -G -v test_pickle and ./python test_bytestrpickle.py pass, but I have not run the entire test suite, as that takes ~90 minutes on my laptop The test script should of co

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: OK, this is the pickle.py patch. A new parameter 'bytestr' has been added to both _Pickler and _Unpickler to toggle the pickle.string<=>bytes behaviour: _Pickler: IF protocol <= 2 AND bytestr=True THEN bytes are stored as STRING/SHORT_BINSTRING/BINSTRING ELS

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : Added file: http://bugs.python.org/file24567/test_bytestrpickle.py ___ Python tracker ___ ___ Python-bugs-list ma

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Antoine Pitrou added the comment: > If I (ever) get the build system and tests of python itself to work, If you have any problems with that, don't hesitate to ask on python-dev (or see http://mail.python.org/mailman/listinfo/core-mentorship ) -- ___

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: Last night, I hacked together a wrapper to do what loewis suggested [1]. It pickles bytes to str (for protocol <= 2), and unpickles str to bytes. If I (ever) get the build system and tests of python itself to work, I'll try and see if I can implement a nice

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Ronny Pfannschmidt added the comment: im unlikely to find the time to try and fix pickle/cpickle myself in the next few months -- nosy: +Ronny.Pfannschmidt ___ Python tracker __

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Éric Araujo : -- versions: +Python 3.3 -Python 2.6, Python 3.1 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsu

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Merlijn van Deen added the comment: Any news on this? Just as a note, pickletools.py also does not reflect the current behaviour; pickle types STRING, BINSTRING and SHORT_BINSTRING are all defined with stack_after=[pystring]: [1, line 992] I(name='STRING', code='S', arg=strin

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Merlijn van Deen : -- nosy: +valhallasw ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.p

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by R. David Murray : -- nosy: +jdharper ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

RonnyPfannschmidt added the comment: i'll try to add some tests now hopefully i can get rid of the implicit badness like trying to coerce bytes to unicode in unpickle and storing bytes as list in pickle for protocol < 3 -- ___ Python tracker

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Gabriel Genellina added the comment: Note that this is also a documentation issue: "The pickle serialization format is guaranteed to be backwards compatible across Python releases." -- assignee: -> georg.brandl components: +Documentation nosy: +gagenellina, georg.brandl

[issue6784] byte/unicode pickle incompatibilities between python2 and python3

Changes by RonnyPfannschmidt : -- title: byte/unicode pickle incompatibilities between python2 andand python3 -> byte/unicode pickle incompatibilities between python2 and python3 ___ Python tracker