[issue25863] ISO-2022 seeking forgets state

[Martin Panter]

Martin Panter added the comment:

After thinking about Issue 26158, I realize the seek() magic numbers don’t 
store any _encoder_ state, only _decoder_ state. That would explain the first 
bug (write, seek, then read). Though for this codec I suspect the decoder state 
is not recorded either, hence the bug with tell().

Personally I don’t care much for seeking text files. But if someone wanted to 
fix the second bug, that might require fixing the incremental decoder’s 
getstate() implementation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25863] ISO-2022 seeking forgets state

[John Walker]

John Walker added the comment:

Here is Martin's message as a unit test. It checks utf-8 and the iso-2022 
family except iso-2022-cn and iso-2022-cn-ext because they are not supported. 
The errors occur with all iso-2022 charsets.

--
keywords: +patch
Added file: http://bugs.python.org/file41451/25863-unittest.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25863] ISO-2022 seeking forgets state

[John Walker]

Changes by John Walker :


--
nosy: +johnwalker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25863] ISO-2022 seeking forgets state

[Ezio Melotti]

Ezio Melotti added the comment:

I confirmed the problem on default (3.6) and verified that it works as expected 
using utf-8 instead of iso-2022-jp.
The code in the above message should be converted into a unittest, and related 
codecs should be checked as well.
The problem is probably in Modules/cjkcodecs/_codecs_iso2022.c

--
components: +Extension Modules
stage: needs patch -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25863] ISO-2022 seeking forgets state

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +serhiy.storchaka
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25863] ISO-2022 seeking forgets state

[Martin Panter]

New submission from Martin Panter:

>>> from io import *
>>> text = TextIOWrapper(BytesIO(), "iso-2022-jp")
>>> text.write(u"P")
1
>>> text.tell()
1
>>> text.write(u"anter 正")
7
>>> text.tell()
12
>>> text.write(u"孝")
1
>>> text.seek(12)
12
>>> text.read()  # Should return 孝, not ASCII
"9'"
>>> text.buffer.getvalue()
b"Panter \x1b$B@59'"
>>> text.seek(1)
1
>>> text.read(7)
'anter 正'
>>> text.tell()  # Another bug?
Traceback (most recent call last):
  File "", line 1, in 
UnicodeDecodeError: 'iso2022_jp' codec can't decode bytes in position 2-3: 
illegal multibyte sequence

--
components: IO, Unicode
messages: 256431
nosy: ezio.melotti, haypo, martin.panter
priority: normal
severity: normal
status: open
title: ISO-2022 seeking forgets state
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com