[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Roundup Robot]

Roundup Robot added the comment:

New changeset cf6e782a7f94 by Serhiy Storchaka in branch '2.7':
Issue #19543: Emit deprecation warning for known non-text encodings.
https://hg.python.org/cpython/rev/cf6e782a7f94

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Roundup Robot]

Roundup Robot added the comment:

New changeset 140a69d950eb by Georg Brandl in branch '3.3':
Issue #20404: reject non-text encodings early in TextIOWrapper.
http://hg.python.org/cpython/rev/140a69d950eb

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is backported to 3.3 patch.

--
nosy: +georg.brandl
Added file: 
http://bugs.python.org/file34221/issue20404_check_valid_textio_codec-3.3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Roundup Robot]

Roundup Robot added the comment:

New changeset f3ec00d2b75e by Nick Coghlan in branch 'default':
Close #20404: blacklist non-text encodings in io.TextIOWrapper
http://hg.python.org/cpython/rev/f3ec00d2b75e

--
nosy: +python-dev
resolution:  -> fixed
stage: commit review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM.

--
stage: test needed -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Nick Coghlan]

Nick Coghlan added the comment:

v3:

- prefix for internal helper APIs is now _PyCodecInfo_ to better distinguish 
them from the ones that take an encoding name
- error check is now just for "is not a text encoding"
- tweaked the name and comment in the error test to be clear that it is codecs 
that are not marked as text encodings that are rejected, not just binary 
transforms
- fixed indentation of multi-line expression in _pyio.py

--
Added file: 
http://bugs.python.org/file33899/issue20404_check_valid_textio_codec_v3.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The _PyCodec_GetIncrementalDecoder name looks too similar to 
PyCodec_IncrementalDecoder. It would be better to use more different name.

And please note my minor comments on Rietveld.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Nick Coghlan]

Nick Coghlan added the comment:

Ah, just noticed the test case is still using the overly specific check for the 
exception wording. I'll fix that, too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Nick Coghlan]

Nick Coghlan added the comment:

Revised patch that avoids doing multiple lookups of the same codec name while 
creating the stream.

Absent any comments, I'll commit this version with appropriate NEWS and What's 
New updates tomorrow.

--
Added file: 
http://bugs.python.org/file33856/issue20404_check_valid_textio_codec_v2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Nick Coghlan]

Nick Coghlan added the comment:

Attached patch checks the requested encoding is a valid text encoding in 
TextIOWrapper.__init__.

Two additional test changes were needed:

- the one that checks handling of non-text-encodings at runtime now tweaks 
quopri to lie about being a text encoding when creating the stream

- the one that checks the interpreter doesn't crash at shutdown needed to be 
adjusted to handle the fact that _pyio now also fails in that situation, but 
with a different error (it can't find the ascii codec because the codec 
machinery is mostly gone)

Currently, this adds a third lookup of the encoding name to the process of 
creating a TextIOWrapper instance. This could be reduced to just one by 
changing the retrieval of the encoder and decoder to look in the retrieved 
codec info tuple, rather than doing the lookup by name again.

--
components: +IO
keywords: +patch
nosy: +pitrou
Added file: 
http://bugs.python.org/file33747/issue20404_check_valid_textio_codec.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Nick Coghlan]

Nick Coghlan added the comment:

I also created issue 20405 as an RFE for 3.5, since it seems there is a 
possible gap in capability relative to Python 2 here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20404] Delayed exception using non-text encodings with TextIOWrapper

[Nick Coghlan]

New submission from Nick Coghlan:

TextIOWrapper doesn't check the codec type early the way the convenience 
methods now do, so the open() method currently still suffers from the problems 
Victor described in issue 19619 for str.encode() and bytes.decode():

>>> with open("hex.txt", 'w') as f:
... f.write("aabbccddeeff")
... 
12
>>> print(open('hex.txt').read())
aabbccddeeff
>>> print(open('hex.txt', encoding='hex').read())
Traceback (most recent call last):
  File "", line 1, in 
TypeError: decoder should return a string result, not 'bytes'

These codecs are never going to work correctly with TextIOWrapper, so we should 
add the appropriate compatibility check in the constructor.

--
assignee: ncoghlan
messages: 209396
nosy: haypo, larry, ncoghlan, serhiy.storchaka
priority: release blocker
severity: normal
stage: test needed
status: open
title: Delayed exception using non-text encodings with TextIOWrapper
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com