[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Benjamin Peterson]

Benjamin Peterson  added the comment:

Python 2 is done.

--
nosy: +benjamin.peterson
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 Python 3 warns even if strings are equal.

Did you mean not equal? In Python 3 strings and bytes are always not equal.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[STINNER Victor]

STINNER Victor added the comment:

Serhiy wrote:
I think that even if we accept this change (I am unsure in this), a warning 
should be raised only when bytes and unicode objects are equal. When they are 
not equal, a warning should not be raised, because this matches Python 3 
behavior.

Python 3 warns even if strings are equal.

$ python3 -b -Wd
Python 3.3.2 (default, Mar  5 2014, 08:21:05) 
e for more information.
 b'abc' == 'abc'
__main__:1: BytesWarning: Comparison between bytes and string
False
 b'abc' == 'abc'
False

The warning is not repeat in the interactive interprter because it is emited 
twice at the same location __main__:1.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think that even if we accept this change (I am unsure in this), a warning 
should be raised only when bytes and unicode objects are equal. When they are 
not equal, a warning should not be raised, because this matches Python 3 
behavior.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Chris Rebert]

Changes by Chris Rebert pyb...@rebertia.com:


--
nosy: +cvrebert

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue19656.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Brett Cannon]

Brett Cannon added the comment:

I thought we gave ourselves the wiggle room to change the warnings we emitted 
for -3 (I unfortunately can't find a reference to something relating to that in 
the Python 2.7 PEP)?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Josh Cogliati]

Josh Cogliati added the comment:

Other than in the source code in Modules/main.c, is -b documented anywhere? 
(For 2.7.6, The html docs, man page, and --help all failed to mention it)

--
nosy: +jrincayc

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[STINNER Victor]

STINNER Victor added the comment:

Attached py2_warn_cmp_bytes_text.patch adds BytesWarning for bytes == unicode, 
bytes != unicode, unicode == bytes, unicode != bytes and similar comparisons 
with bytearray. The new warnings are added when -b or -bb command line options 
are used.

As a consequence, a lot of tests are failing with the patch applied and -bb 
command line option.

Some tests are obviously wrong (unicode expected, but the tests use bytes), but 
it's much more complex to fix tricky modules like urllib, os.path, json and re 
(sre_parse) to handle correctly bytes and unicode. In some other cases, the 
warning should be made quiet because a same test compares bytes and then text.

It also means that programs currently working fine with Python 2.7.6 with -3 
-b options will start to see new BytesWarning warnings. Is it acceptable?

What is the purpose of -b in Python 2? Help developers to notice earlier future 
Unicode issues in their program, or help them to port their code to Python 3?

Maybe the new warnings should only by emited if -3 and -b options are used at 
the same time?

Tell me if you would like to see my work-in-progress patch to fix the whole 
test suite. Just the stats:

$ hg diff --stat
 Lib/_pyio.py |  16 
 Lib/ctypes/test/test_arrays.py   |  12 ++--
 Lib/ctypes/test/test_buffers.py  |  20 ++--
 Lib/ctypes/test/test_cast.py |   2 +-
 Lib/ctypes/test/test_memfunctions.py |  10 +-
 Lib/ctypes/test/test_prototypes.py   |  10 +-
 Lib/ctypes/test/test_structures.py   |   2 +-
 Lib/fractions.py |   1 +
 Lib/sqlite3/dump.py  |   8 
 Lib/sqlite3/test/dump.py |   4 ++--
 Lib/sre_parse.py |   3 ++-
 Lib/test/string_tests.py |   6 +++---
 Lib/test/test_builtin.py |  14 ++
 Lib/test/test_bytes.py   |  27 +--
 Lib/test/test_format.py  |  10 +++---
 Lib/test/test_future4.py |   2 +-
 Lib/test/test_pyexpat.py |  28 ++--
 Lib/test/test_sax.py |  20 ++--
 Lib/test/test_tempfile.py|   4 ++--
 Objects/bytearrayobject.c|   2 +-
 Objects/stringobject.c   |   9 +
 Objects/unicodeobject.c  |   8 
 22 files changed, 135 insertions(+), 83 deletions(-)

A funny one:

diff -r 670fb496f1f6 Lib/test/test_future4.py
--- a/Lib/test/test_future4.py  Sun May 11 23:37:26 2014 -0400
+++ b/Lib/test/test_future4.py  Tue May 13 03:28:12 2014 +0200
@@ -43,5 +43,5 @@ class TestFuture(unittest.TestCase):
 def test_main():
 test_support.run_unittest(TestFuture)
 
-if __name__ == __main__:
+if __name__ == b__main__:
 test_main()

--
keywords: +patch
Added file: http://bugs.python.org/file35235/py2_warn_cmp_bytes_text.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[STINNER Victor]

STINNER Victor added the comment:

The title of the issue is python2 -3 does not warn about str/unicode to bytes 
conversions and comparisons.

IMO it would be insane to emit BytesWarning on unicode(str). It would break 
most code using unicode. six.u() function is based on this feature. For 
example, six.u(abc) calls unicode(abc) in Python 2.

I have no opinion for the encode operation: str(unicode).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Brett Cannon]

Brett Cannon added the comment:

Yes, that's a possibility if we want to take the route and essentially prevent 
people from ever explicitly knowing that a str in Python 2 will be a str in 
Python 3 and they are okay with that.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Brett Cannon]

Brett Cannon added the comment:

Unfortunately it's impossible to warn against this in Python 2 since the bytes 
type is just another name for the str type:

 str == bytes
True
 type(b'1')
type 'str'

What we could potentially do, though, is change things such that -3 does what 
you are after when comparing bytes/str to unicode in Python 2. Unfortunately in 
that instance it's still a murky question as to whether that will help things 
more than hurt them as some people explicitly leave strings as-is in both 
Python 2 and Python 3 for either speed or code simplicity reasons.

--
nosy: +brett.cannon
title: python2 -3 does not warn about str to bytes conversions and comparisons 
- python2 -3 does not warn about str/unicode to bytes conversions and 
comparisons

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21401] python2 -3 does not warn about str/unicode to bytes conversions and comparisons

[Joshua J Cogliati]

Joshua J Cogliati added the comment:

Hm.  That is a good point.  Possibly it could only be done when 
from __future__ import unicode_literals
has been used.  For example:

python2 -3
Python 2.7.5 snip
Type help, copyright, credits or license for more information.
 type(ba) == type(a)
True
 from __future__ import unicode_literals
 type(ba) == type(a)
False
 ba == a
True
 ba + a
u'aa'
 


After unicode_literals is used, then ba and a have a different type and the 
same code would be an issue in python3:
 python3
Python 3.3.2 snip
 type(ba) == type(a)
False
 ba == a
False
 ba + a
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: can't concat bytes to str


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21401
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com