[issue11638] pysetup un sdist crashes with weird trace if version is unicode by accident
Changes by RonnyPfannschmidt : Added file: http://bugs.python.org/file21344/hgdistver.py ___ Python tracker <http://bugs.python.org/issue11638> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11638] pysetup un sdist crashes with weird trace if version is unicode by accident
RonnyPfannschmidt added the comment: actually its enough to have the version_hook set the version to u'0.0' -- ___ Python tracker <http://bugs.python.org/issue11638> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11638] pysetup un sdist crashes with weird trace if version is unicode by accident
Changes by RonnyPfannschmidt : Added file: http://bugs.python.org/file21343/setup.cfg ___ Python tracker <http://bugs.python.org/issue11638> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11638] pysetup un sdist crashes with weird trace if version is unicode by accident
RonnyPfannschmidt added the comment: here the file that passed in the unicode string via hook note that all of the passed characters are actually ascii -- Added file: http://bugs.python.org/file21342/hgdistver.py ___ Python tracker <http://bugs.python.org/issue11638> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11638] pysetup un sdist crashes with weird trace if version is unicode by accident
New submission from RonnyPfannschmidt : i passed in a unicode value as version by accident, resulted in: Traceback (most recent call last): File "/home/ronny/.local/venvs/clean/bin/pysetup", line 7, in execfile(__file__) File "/home/ronny/Projects/distutils2/distutils2/pysetup", line 5, in main() File "/home/ronny/Projects/distutils2/distutils2/run.py", line 486, in main return dispatcher() File "/home/ronny/Projects/distutils2/distutils2/run.py", line 477, in __call__ return func(self, self.args) File "/home/ronny/Projects/distutils2/distutils2/run.py", line 166, in _run dist.run_command(cmd, dispatcher.command_options[cmd]) File "/home/ronny/Projects/distutils2/distutils2/dist.py", line 781, in run_command cmd_obj.run() File "/home/ronny/Projects/distutils2/distutils2/command/sdist.py", line 183, in run self.make_distribution() File "/home/ronny/Projects/distutils2/distutils2/command/sdist.py", line 327, in make_distribution owner=self.owner, group=self.group) File "/home/ronny/Projects/distutils2/distutils2/command/cmd.py", line 426, in make_archive owner=owner, group=group) File "/home/ronny/Projects/distutils2/distutils2/_backport/shutil.py", line 588, in make_archive filename = func(base_name, base_dir, **kwargs) File "/home/ronny/Projects/distutils2/distutils2/_backport/shutil.py", line 426, in _make_tarball tar = tarfile.open(archive_name, 'w|%s' % tar_compression[compress]) File "/home/ronny/Projects/distutils2/distutils2/_backport/tarfile.py", line 1693, in open _Stream(name, filemode, comptype, fileobj, bufsize), File "/home/ronny/Projects/distutils2/distutils2/_backport/tarfile.py", line 434, in __init__ self._init_write_gz() File "/home/ronny/Projects/distutils2/distutils2/_backport/tarfile.py", line 462, in _init_write_gz self.__write(self.name + NUL) File "/home/ronny/Projects/distutils2/distutils2/_backport/tarfile.py", line 478, in __write self.buf += s UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 1: ordinal not in range(128) note that i have no idea where the 0x8b is from, if i just pass the version trough str it works (which means something is wrong somewhere else, unicode just triggers it) -- assignee: tarek components: Distutils2 messages: 131781 nosy: RonnyPfannschmidt, alexis, eric.araujo, tarek priority: normal severity: normal status: open title: pysetup un sdist crashes with weird trace if version is unicode by accident versions: 3rd party ___ Python tracker <http://bugs.python.org/issue11638> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
RonnyPfannschmidt added the comment: i'll try to add some tests now hopefully i can get rid of the implicit badness like trying to coerce bytes to unicode in unpickle and storing bytes as list in pickle for protocol < 3 -- ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and python3
Changes by RonnyPfannschmidt : -- title: byte/unicode pickle incompatibilities between python2 andand python3 -> byte/unicode pickle incompatibilities between python2 and python3 ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and and python3
RonnyPfannschmidt added the comment: in case the actual behavior is not supposed to change how about a way to declare one wants exact 1:1 mapping between py2<>py3, so str<>bytes and unicode<>str will work for sure something like load/dump(..., encoding=bytes) just crossed my mind -- ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and and python3
RonnyPfannschmidt added the comment: unpickle of any non-ascii string from python2 will break the only way out would be to ensure text strings and a single defined encoding (at that point storing unicode strings in any case seems more practical) also byte-strings stored as python2 str would break and since i pass around binary strings as parts of objects, its just completely broken for me -- ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and and python3
RonnyPfannschmidt added the comment:
its even worse
python3:
>>> import pickle
>>> pickle.dumps(b'', protocol=2)
b'\x80\x02c__builtin__\nbytes\nq\x00]q\x01\x85q\x02Rq\x03.'
python2.6:
>>> import pickle
>>> pickle.loads('\x80\x02c__builtin__\nbytes\nq\x00]q\x01\x85q\x02Rq\x03.')
'[]'
--
___
Python tracker
<http://bugs.python.org/issue6784>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and and python3
RonnyPfannschmidt added the comment: Since it breaks for anything non-ascii, its not that helpfull after all and since python2 strings are encoding-unaware there is no way to fix it. It might be preferable to supply unpicklers that are cappable of coercing if the user really wants wants coercing. yup > > > 3. python 3 string map to python 2 unicode > > That's also the case, AFAICT. yup > > > 4. python 3 bytestring maps to python 2 string > > Hmm. This may be indeed a mistake. Until r61467, bytes were saved > with the (BIN)STRING code; not sure why this was changed. Python 3 is indeed evil there. b'\x80\x02c__builtin__\nbytes\nq\x00]q\x01\x85q\x02Rq\x03.' I'm convinced that a 1:1 mapping of python2 string from/to python3 bytestrings is the least surprising behaviour and will keep surprising errors away when needing to communicate between different python versions. It just has bitten me, and i suspect will will get others, too. Unpickle that completely fails in the face of encodings is not desirable at all. -- ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and and python3
RonnyPfannschmidt added the comment: the basic behavior i want to see for all protocols <= 2 1. python 2 string maps to python3 byte-string 2. python 2 unicode maps to python3 string 3. python 3 string map to python 2 unicode 4. python 3 bytestring maps to python 2 string anything else is is confusing and may break for example one can't unpickle '\xFF' in python3 if it was pickled in python2 note that these changes seem irrelevant for protocol 3 as python2.x doesn't support it -- ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] byte/unicode pickle incompatibilities between python2 and and python3
Changes by RonnyPfannschmidt : -- title: bytw/unicode string incompatibilities between python2 and and python3 -> byte/unicode pickle incompatibilities between python2 and and python3 ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6784] bytw/unicode string incompatibilities between python2 and and python3
New submission from RonnyPfannschmidt : i just noticed that there are some slight differences of the bytestring/unicodestring pickles between python2/3 using the protocols 0, 1 and 2 the first things i noticed are: a str from python2 is unpickled as unicode in python3 (fails for byte strings that don't fit whats expected for unicode) a bytes instance from python3 is pickled as custom class in protocols <3 i'll write a script to try all combinations of protocols and string variations and transfer directions -- components: None messages: 91966 nosy: RonnyPfannschmidt severity: normal status: open title: bytw/unicode string incompatibilities between python2 and and python3 type: behavior versions: Python 2.6, Python 3.1 ___ Python tracker <http://bugs.python.org/issue6784> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
