[issue12508] Codecs Anomaly

Change by Irit Katriel : -- versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.2, Python 3.3 ___ Python tracker ___ _

[issue12508] Codecs Anomaly

Changes by Serhiy Storchaka : -- versions: +Python 2.7, Python 3.3 Added file: http://bugs.python.org/file26013/fffd-2.py ___ Python tracker ___ _

[issue12508] Codecs Anomaly

Ezio Melotti added the comment: AFAIU final means: * final=0: I'm passing in a few bytes, but there are more to come, so if the last byte(s) doesn't make sense on its own (e.g. it's a start byte but the continuation bytes are missing), wait for the others before raising an error; * final=1

[issue12508] Codecs Anomaly

Marc-Andre Lemburg added the comment: The final parameter is an extension to the decoder API signature, so it's not surprising that not all codecs implement it. The ones that do should use it for all calls, since that way the actual consumed number of bytes is correctly reported back to the Str

[issue12508] Codecs Anomaly

Changes by Ezio Melotti : -- nosy: +lemburg ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue12508] Codecs Anomaly

Ezio Melotti added the comment: IIUC this happens because StreamReader calls codecs.utf_8_decode without passing final=1 [0], so when the decoder finds the trailing F4 it doesn't decode it yet because it waits from the other 3 bytes (F4 is the start byte of a 4-bytes UTF-8 sequence): >>> b =

[issue12508] Codecs Anomaly

Changes by Raymond Hettinger : -- priority: normal -> high ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http

[issue12508] Codecs Anomaly

STINNER Victor added the comment: You should use the io module, it doesn't have the bug :) -- ___ Python tracker ___ ___ Python-bugs-

[issue12508] Codecs Anomaly

STINNER Victor added the comment: I confirm, there is a bug in codecs.StreamReader. -- nosy: +haypo ___ Python tracker ___ ___ Python

[issue12508] Codecs Anomaly

Changes by Ezio Melotti : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.py

[issue12508] Codecs Anomaly

New submission from Saul Spatz : The attached script produces the output 'A\ufffdBC\ufffd' 'A\ufffdBC' although it seems to me that both lines should be the same. The first line is correct, I think, since the at the end is a maximal subpart of an ill-formed subsequence, according to the de