New submission from Mikhail Korobov <[email protected]>:
Hi,
I think that the way PyPy implements support for custom encoding/decoding error
handlers is not compatible
with CPython. An example script that reproduces this is in attachements.
CPython 2.6.7 agrees with CPython 3.3.2:
$ python3.3 error_handlers.py
\x80\xa3
UnicodeDecodeError('utf-8', b'\x80\xa3', 0, 1, 'invalid start byte')
UnicodeDecodeError('utf-8', b'\x80\xa3', 0, 1, 'invalid start byte')
!!
\xe3\xa3
UnicodeDecodeError('utf-8', b'\xe3\xa3', 0, 2, 'unexpected end of data')
UnicodeDecodeError('utf-8', b'\xe3\xa3', 0, 2, 'unexpected end of data')
!!
$ python2.6 error_handlers.py
\x80\xa3
UnicodeDecodeError('utf8', '\x80\xa3', 0, 1, 'invalid start byte')
UnicodeDecodeError('utf8', '\x80\xa3', 0, 1, 'invalid start byte')
!!
\xe3\xa3
UnicodeDecodeError('utf8', '\xe3\xa3', 0, 2, 'unexpected end of data')
UnicodeDecodeError('utf8', '\xe3\xa3', 0, 2, 'unexpected end of data')
!!
PyPy 2.0.2 produces different results (different exceptions are raised, the
output
is also not the same):
$ pypy error_handlers.py
\x80\xa3
UnicodeDecodeError('utf-8', '\x80\xa3', 0, 1, 'invalid start byte')
UnicodeDecodeError('utf-8', '\x80\xa3', 1, 2, 'invalid start byte')
!!
\xe3\xa3
UnicodeDecodeError('utf-8', '\xe3\xa3', 0, 2, 'unexpected end of data')
!
----------
messages: 5922
nosy: kmike, pypy-issue
priority: bug
status: unread
title: custom encoding/decoding error handlers are not compatible with CPython
________________________________________
PyPy bug tracker <[email protected]>
<https://bugs.pypy.org/issue1536>
________________________________________
_______________________________________________
pypy-issue mailing list
[email protected]
http://mail.python.org/mailman/listinfo/pypy-issue
