[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread Gregory P. Smith


Gregory P. Smith  added the comment:

OverflowError is now raised for negative values that would trigger a problem 
and the unittest has been updated to test this.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset d5f017bbd65f37ac53fd3c6c439a53155eef2475 by Miss Islington (bot) 
in branch '3.7':
bpo-13312: Avoid int underflow in time year. (GH-8912)
https://github.com/python/cpython/commit/d5f017bbd65f37ac53fd3c6c439a53155eef2475


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset c47acc2bb1d0a3fb6dda14ced958d272fb2821a6 by Miss Islington (bot) 
in branch '3.6':
bpo-13312: Avoid int underflow in time year. (GH-8912)
https://github.com/python/cpython/commit/c47acc2bb1d0a3fb6dda14ced958d272fb2821a6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34494] simple "sequence" class ignoring __len__

2018-08-24 Thread Tyler Reddy


New submission from Tyler Reddy :

Downstream in NumPy we've noticed that a "sequence" object defined as below 
will hang (infinite __getitem__ calls) if we try to turn it into an array. The 
same holds in CPython for converting it to a list:

class OneList:

def __len__(self):
# this won't be checked by
# PySequence_Fast and several
# over C API calls
return 1

def __getitem__(self, x):
# called indefinitely by
# i.e., PySequence_Fast
return 1

Just to confirm -- this is intentional / desired behavior:

list(OneList()) should hang in CPython?

related: 
https://github.com/numpy/numpy/issues/8912
https://github.com/numpy/numpy/pull/11815
https://stackoverflow.com/a/43566241/2942522

--
messages: 324037
nosy: treddy
priority: normal
severity: normal
status: open
title: simple "sequence" class ignoring __len__
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34491] _bsddb.c: Missing Py_DECREF() in DB_join()

2018-08-24 Thread Xiang Zhang


Change by Xiang Zhang :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34491] _bsddb.c: Missing Py_DECREF() in DB_join()

2018-08-24 Thread Xiang Zhang


New submission from Xiang Zhang :


New changeset 032e85f3fb6d2e6668d4e860b40525ed6cb8dd69 by Xiang Zhang (Zackery 
Spytz) in branch '2.7':
bpo-34491: Add missing Py_DECREF() in _bsddb's DB_join() (GH-8909)
https://github.com/python/cpython/commit/032e85f3fb6d2e6668d4e860b40525ed6cb8dd69


--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34442] zlib module not built on windows

2018-08-24 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

What precisely does 'running' mean, so that we can try to reproduce?

python -m test.test_zipfile

Runs 208/207 tests in about a minute on the repository debug builds without 
failure for both 3.8 and 3.7.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33550] Sigpipe handling issue should be documented

2018-08-24 Thread Mariatta Wijaya


Mariatta Wijaya  added the comment:

Thanks!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33550] Sigpipe handling issue should be documented

2018-08-24 Thread miss-islington


Change by miss-islington :


--
keywords: +patch
pull_requests: +8389
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33550] Sigpipe handling issue should be documented

2018-08-24 Thread Mariatta Wijaya


Mariatta Wijaya  added the comment:


New changeset 7b0ed43af55c1e2844aa0ccd5e088b2ddd38dbdb by Mariatta (Miss 
Islington (bot)) in branch '3.7':
bpo-33550: Warn not to set SIGPIPE to SIG_DFL (GH-6773)
https://github.com/python/cpython/commit/7b0ed43af55c1e2844aa0ccd5e088b2ddd38dbdb


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34429] On Windows tempfile.TemporaryFile behaves like NamedTemporaryFile

2018-08-24 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

3.4 and 3.5 only get security fixes.

Anyone preparing PR should check that behavior and doc is same on master; I 
just assumed in setting Versions header.

In the proposed text, it is not clear that 'non' applies to cygwin also.  
Instead: "On platforms that are neither Posix nor Cygwin, ..."
Or "On non-Posix, non-Cygwin platforms, ..."

--
nosy: +terry.reedy
versions: +Python 3.8 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread Mariatta Wijaya


Mariatta Wijaya  added the comment:


New changeset b26fcd6e22b7c150318d555f67b009cedcf5c797 by Mariatta (Miss 
Islington (bot)) in branch '3.6':
bpo-34426: fix typo (__lltrace__ -> __ltrace__) (GH-8822)
https://github.com/python/cpython/commit/b26fcd6e22b7c150318d555f67b009cedcf5c797


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset 0dc75f04575a6bfb2b072f62e5233444aea23e89 by Miss Islington (bot) 
in branch '3.7':
bpo-34426: fix typo (__lltrace__ -> __ltrace__) (GH-8822)
https://github.com/python/cpython/commit/0dc75f04575a6bfb2b072f62e5233444aea23e89


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34428] tokenize

2018-08-24 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

Luke: If and when you have legal code that fails when run with purely with 
CPython, you can re-open or try an new issue.

--
nosy: +serhiy.storchaka, terry.reedy
resolution:  -> third party
stage:  -> resolved
status: open -> closed
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread Mariatta Wijaya


Mariatta Wijaya  added the comment:

Thanks!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8388

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8387

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread Mariatta Wijaya


Mariatta Wijaya  added the comment:


New changeset 09efe49c07e2d5f93b415ead757c87e20cc0026f by Mariatta (Danish 
Prakash) in branch 'master':
bpo-34426: fix typo (__lltrace__ -> __ltrace__) (GH-8822)
https://github.com/python/cpython/commit/09efe49c07e2d5f93b415ead757c87e20cc0026f


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34427] calling MutableSequence.extend on self produces infinite loop

2018-08-24 Thread Terry J. Reedy


Change by Terry J. Reedy :


--
versions: +Python 3.6, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34426] "__lltrace__" seems to be wrong , "__ltrace__" is correct in Misc/SpecialBuilds.txt

2018-08-24 Thread Terry J. Reedy


Change by Terry J. Reedy :


--
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8386

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8385

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread Gregory P. Smith


Gregory P. Smith  added the comment:


New changeset 76be0f8b7dbe649ad4821144461800ffb0d0 by Gregory P. Smith in 
branch 'master':
bpo-13312: Avoid int underflow in time year. (GH-8912)
https://github.com/python/cpython/commit/76be0f8b7dbe649ad4821144461800ffb0d0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33569] dataclasses InitVar does not maintain any type info

2018-08-24 Thread Eric V. Smith


Eric V. Smith  added the comment:

test message, please ignore

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31844] HTMLParser: undocumented not implemented method

2018-08-24 Thread Ezio Melotti


Change by Ezio Melotti :


--
assignee:  -> ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34453] ipaddress module accepts some strange IPv6 addresses that it shouldn't

2018-08-24 Thread Michał Radwański

Michał Radwański  added the comment:

According to the RFC ( https://tools.ietf.org/html/rfc4291#section-2.2 ), last 
four bytes can be an address in the format of IPv4 address. The latter doesn't 
specify whether leading zeros are permitted or not. For illustration let's look 
at the implementation of the POSIX function `inet_aton` from Fedora 28, as 
documented by manual:

> In  all  of the above forms, components of the dotted address can be 
> specified in decimal, octal (with a leading 0), or hexadecimal, with a 
> leading 0X).

But notice that if you prefix a 0 with 0, the value stays the same, so it 
doesn't hurt not to raise errors if given either of 

::::::00.00.00.00
::::::000.000.000.000

If however you try

::::::000.000.010.000

then the address is ambiguous, so appropriately you get 

AddressValueError: Ambiguous (octal/decimal) value in '010' not permitted in 
'000.000.010.000' in '::::::000.000.010.000'


I believe the issue is handled correctly.

--
nosy: +enedil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread Gregory P. Smith


Change by Gregory P. Smith :


--
pull_requests: +8384
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13312] test_time fails: strftime('%Y', y) for negative year

2018-08-24 Thread Gregory P. Smith


Change by Gregory P. Smith :


--
assignee:  -> gregory.p.smith
nosy: +gregory.p.smith
versions: +Python 3.6, Python 3.7, Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1621] Do not assume signed integer overflow behavior

2018-08-24 Thread Gregory P. Smith


Change by Gregory P. Smith :


--
versions: +Python 3.7, Python 3.8 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34492] Python/coreconfig.c: Missing NULL check in _Py_wstrlist_copy()

2018-08-24 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset eb746dbae8b320758ee08f811316d7f283435cc0 by Victor Stinner 
(Alexey Izbyshev) in branch 'master':
bpo-34492: Python/coreconfig.c: Fix _Py_wstrlist_copy() (GH-8910)
https://github.com/python/cpython/commit/eb746dbae8b320758ee08f811316d7f283435cc0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34493] Objects/genobject.c: Missing NULL check in compute_cr_origin()

2018-08-24 Thread Alexey Izbyshev


Change by Alexey Izbyshev :


--
keywords: +patch
pull_requests: +8383
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34493] Objects/genobject.c: Missing NULL check in compute_cr_origin()

2018-08-24 Thread Alexey Izbyshev


New submission from Alexey Izbyshev :

The return value of PyTuple_New() is not checked for NULL at 
https://github.com/python/cpython/blob/ef8861c112ed1dac9351958c121bc24ca4ecdb08/Objects/genobject.c#L1130
 and then dereferenced.

--
components: Interpreter Core
messages: 324022
nosy: berker.peksag, izbyshev, njs, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Objects/genobject.c: Missing NULL check in compute_cr_origin()
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34492] Python/coreconfig.c: Missing NULL check in _Py_wstrlist_copy()

2018-08-24 Thread Alexey Izbyshev


Change by Alexey Izbyshev :


--
keywords: +patch
pull_requests: +8382
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34492] Python/coreconfig.c: Missing NULL check in _Py_wstrlist_copy()

2018-08-24 Thread Alexey Izbyshev


New submission from Alexey Izbyshev :

The return value of PyMem_RawMalloc() is not checked for NULL at 
https://github.com/python/cpython/blob/ef8861c112ed1dac9351958c121bc24ca4ecdb08/Python/coreconfig.c#L71.

Reported by Svace static analyzer.

--
components: Interpreter Core
messages: 324021
nosy: berker.peksag, izbyshev, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Python/coreconfig.c: Missing NULL check in _Py_wstrlist_copy()
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6531] atexit_callfuncs() crashing within Py_Finalize() when using multiple interpreters.

2018-08-24 Thread Eric Snow


Change by Eric Snow :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34453] ipaddress module accepts some strange IPv6 addresses that it shouldn't

2018-08-24 Thread Luke Kenneth Casson Leighton


Luke Kenneth Casson Leighton  added the comment:

hi prudvi: i have absolutely no idea.  i am simply running test
validators online, which show and confirm that they are correctly
INVALID.  a google search shows a number of IPv6 validators:
https://www.google.co.uk/search?q=ipv6+address+validator

i have absolutely no actual knowledge of what constitutes a valid IPv6
address or not, nor know of any "official" resources.

all i know is: consensus tends to agree that these two addresses
are accepted by the python ipaddress module when they should not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34491] _bsddb.c: Missing Py_DECREF() in DB_join()

2018-08-24 Thread Zackery Spytz


Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +8380
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34491] _bsddb.c: Missing Py_DECREF() in DB_join()

2018-08-24 Thread Zackery Spytz


Change by Zackery Spytz :


--
components: Extension Modules
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: _bsddb.c: Missing Py_DECREF() in DB_join()
type: resource usage
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34461] Availability of parsers in etree initializer

2018-08-24 Thread Stefan Behnel


Stefan Behnel  added the comment:

The normal way to use ElementTree is to import the ElementTree module from the 
xml.etree package. Why do you think that needs to change? And why do you want 
to enforce an import of all modules before you even know which of them the 
users actually wants?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34465] ipaddress should accept bytearray in addition to bytes

2018-08-24 Thread Prudvi RajKumar Maddala


Change by Prudvi RajKumar Maddala :


--
keywords: +patch
pull_requests: +8379
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34490] transport.get_extra_info('sockname') of test_asyncio fails on AIX

2018-08-24 Thread Michael Felt


Change by Michael Felt :


--
keywords: +patch
pull_requests: +8377
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34490] transport.get_extra_info('sockname') of test_asyncio fails on AIX

2018-08-24 Thread Michael Felt


Change by Michael Felt :


--
keywords: +patch, patch
pull_requests: +8377, 8378
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34490] transport.get_extra_info('sockname') of test_asyncio fails on AIX

2018-08-24 Thread Michael Felt


New submission from Michael Felt :

Inspired by msg211764, issue20682

In short: 
   For address family of AF_UNIX or AF_UNIX_CCSID, getsockname() returns 0 if 
issued before a bind(). The address length is 0. This is always the case for 
sockets created by socketpair().

That is at least one possible explanation I found.

Have tested on AIX 6.1 and AIX 7.1 - transport.get_extra_info('sockname') 
returns None

--
components: Tests
messages: 324017
nosy: Michael.Felt
priority: normal
severity: normal
status: open
title: transport.get_extra_info('sockname') of test_asyncio fails on AIX
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20682] test_create_ssl_unix_connection() of test_asyncio failed on "x86 Tiger 3.x"

2018-08-24 Thread Michael Felt


Michael Felt  added the comment:

Ah, leave closed. I'll make a new issue, and reference this one. Sorry for the 
noise here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue20682] test_create_ssl_unix_connection() of test_asyncio failed on "x86 Tiger 3.x"

2018-08-24 Thread Michael Felt


Michael Felt  added the comment:

Inspired by msg211764 I made a PR - which I hope is okay to add here.

If not, a new issue and PR can be made later.

In short: 
   For address family of AF_UNIX or AF_UNIX_CCSID, getsockname() returns 0 if 
issued before a bind(). The address length is 0. This is always the case for 
sockets created by socketpair().

That is at least one possible explanation I found.

Have testewd on AIX 6.1 and AIX 7.1 - on AF_UNIX family getsockname() returns 
'None' (i.e., transport.get_extra_info('sockname') returns None)

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34453] ipaddress module accepts some strange IPv6 addresses that it shouldn't

2018-08-24 Thread Prudvi RajKumar Maddala


Prudvi RajKumar Maddala  added the comment:

Hi lkcl, can you help me understand why exactly the below IPv6 addresses are 
invalid?
('::::::00.00.00.00', False)
('::::::000.000.000.000', False)

I pulled the above addresses from your git testcase.

But, below 2 IPv6 addresses are given as valid
('::::::123.123.123.123', True)
('::::::123.123.123.123', True)

I don't see any difference between both. Both of them have IPv4 styled 
suffixes. According to the code, they are getting converted at runtime. Can you 
please clarify? 

Thanks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34489] subprocess: execution of batch-files (.cmd/.bat) is vulnerable in python for windows / insufficient escape

2018-08-24 Thread Sergey G. Brester

New submission from Sergey G. Brester :

There is a vulnerability "insufficient escape of special chars for quoting of 
arguments by exec process" for python-language, if executing windows 
batch-files (bat/cmd).

### What version of python is affected?
All

### Does this issue reproduce with the latest master?
Yes

### What did you do?
Execution of batch-file using `subprocess` module with arguments containing 
some special meta-characters.

A recipe for reproducing the error as well as more extensive PoC with 
additional info (and more lang's affected also):
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/README.md
A complete runnable program:
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/test-dump-inv.py

 A simple example:
```diff
 # invoke exe-file:
 >>> import subprocess
 >>> subprocess.call(['test-dump.exe', 'test'])
+`test-dump.exe´ `test´
 # invoke cmd-file:
 >>> subprocess.call(['test-dump.CMD', 'test'])
-`test-dump.exe´ `test´my_domain\sebres
```
For more "broken" cases, see the result of my test-suite:
- 
https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/results/python.diff

### What did you expect to see?
Arguments are escaped/quoted properly.

### What did you see instead?
Arguments are insufficient escaped/quoted, so it is vulnerable currently.

### Solution:

For possible solution see the github-PR#8906:
- https://github.com/python/cpython/pull/8906 
For algorithm description:
- https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/SOLUTION.md
resp. how it was fixed in TCL (C-code):
- https://core.tcl-lang.org/tcl/vdiff?from=core-8-5-branch=0-day-21b0629c81 
(see the function `BuildCommandLine`)

### Possible similar issues:
bpo-33515

--
components: Library (Lib), Windows
messages: 324013
nosy: paul.moore, sebres, steve.dower, tim.golden, zach.ware
priority: normal
pull_requests: 8376
severity: normal
status: open
title: subprocess: execution of batch-files (.cmd/.bat) is vulnerable in python 
for windows / insufficient escape
type: security
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33515] subprocess.Popen on a Windows batch file always acts as if shell=True

2018-08-24 Thread Sergey G. Brester


Change by Sergey G. Brester :


--
keywords: +patch
pull_requests: +8375
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34470] windows msi in headless mode fails to install Script directory on reinstall if the python folder was deleted but not uninstalled

2018-08-24 Thread Christopher Knorowski


Christopher Knorowski  added the comment:

This also happens in python 3.7. following a similar procedure.

On Wed, Aug 22, 2018 at 9:01 PM Zachary Ware  wrote:

>
> Zachary Ware  added the comment:
>
> Python 2.7 is nearing end-of-life, its installer scheme is no longer used
> on any other branch, and the maintainer of that installer retired some
> years ago.  If you provide a patch, we can take a look and possibly merge
> it for you, but it's unlikely that this will be picked up by a maintainer
> without a patch to review.
>
> If you would like to try patching it, all the necessary files should be in
> 2.7/Tools/msi, and you can feel free to ask questions here if you get
> stuck.  Good luck!
>
> --
> components: +Windows
> nosy: +paul.moore, steve.dower, tim.golden, zach.ware
> priority: normal -> low
> type: compile error -> behavior
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34448] incomplete/missing message from configure.ac for usable wchar_t

2018-08-24 Thread Berker Peksag


Change by Berker Peksag :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34488] improve performance of BytesIO.writelines() by avoiding creation of unused PyLongs

2018-08-24 Thread Sergey Fedoseev


Change by Sergey Fedoseev :


--
keywords: +patch
pull_requests: +8374
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34488] improve performance of BytesIO.writelines() by avoiding creation of unused PyLongs

2018-08-24 Thread Sergey Fedoseev


New submission from Sergey Fedoseev :

Currently BytesIO.writelines() uses BytesIO.write() which returns number of 
written bytes as Python ints, but these ints are not used by 
BytesIO.writelines(), so avoiding creation of these ints can improve 
performance of BytesIO.writelines() especially for a large number of small 
lines.

Benchmarks:

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s 
"from io import BytesIO; lines = [b'']*1;" "BytesIO().writelines(lines)"
  
/home/sergey/tmp/cpython-master-venv/bin/python: . 153 us 
+- 3 us
/home/sergey/tmp/cpython-venv/bin/python: . 126 us +- 4 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 153 us +- 3 
us -> [/home/sergey/tmp/cpython-venv/bin/python] 126 us +- 4 us: 1.22x faster 
(-18%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s 
"from io import BytesIO; lines = [b'b']*1;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: . 164 us 
+- 2 us
/home/sergey/tmp/cpython-venv/bin/python: . 142 us +- 1 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 164 us +- 2 
us -> [/home/sergey/tmp/cpython-venv/bin/python] 142 us +- 1 us: 1.16x faster 
(-13%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s 
"from io import BytesIO; lines = [b'b'*10]*1;" "BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: . 387 us 
+- 6 us
/home/sergey/tmp/cpython-venv/bin/python: . 365 us +- 8 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 387 us +- 6 
us -> [/home/sergey/tmp/cpython-venv/bin/python] 365 us +- 8 us: 1.06x faster 
(-5%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s 
"from io import BytesIO; lines = [b'b'*100]*1;" 
"BytesIO().writelines(lines)"   
 
/home/sergey/tmp/cpython-master-venv/bin/python: . 319 us 
+- 5 us
/home/sergey/tmp/cpython-venv/bin/python: . 305 us +- 16 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 319 us +- 5 
us -> [/home/sergey/tmp/cpython-venv/bin/python] 305 us +- 16 us: 1.04x faster 
(-4%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s 
"from io import BytesIO; lines = [b'b'*1000]*1;" 
"BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: . 1.13 ms 
+- 0.02 ms
/home/sergey/tmp/cpython-venv/bin/python: . 988 us +- 88 us

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 1.13 ms +- 
0.02 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 988 us +- 88 us: 1.14x 
faster (-12%)

python -m perf timeit --compare-to ~/tmp/cpython-master-venv/bin/python -s 
"from io import BytesIO; lines = [b'b'*1]*1;" 
"BytesIO().writelines(lines)"
/home/sergey/tmp/cpython-master-venv/bin/python: . 38.7 ms 
+- 0.1 ms
/home/sergey/tmp/cpython-venv/bin/python: . 38.2 ms +- 0.1 
ms

Mean +- std dev: [/home/sergey/tmp/cpython-master-venv/bin/python] 38.7 ms +- 
0.1 ms -> [/home/sergey/tmp/cpython-venv/bin/python] 38.2 ms +- 0.1 ms: 1.01x 
faster (-1%)

--
components: Extension Modules
messages: 324011
nosy: sir-sigurd
priority: normal
severity: normal
status: open
title: improve performance of BytesIO.writelines() by avoiding creation of 
unused PyLongs
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34448] incomplete/missing message from configure.ac for usable wchar_t

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset 4973f767891e8eb14925a58313085e14cf6ea621 by Miss Islington (bot) 
in branch '3.7':
bpo-34448: Improve output of usable wchar_t check (GH-8846)
https://github.com/python/cpython/commit/4973f767891e8eb14925a58313085e14cf6ea621


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31861] add aiter() and anext() functions to operator module

2018-08-24 Thread Joshua Bronson


Joshua Bronson  added the comment:

Interesting, thanks Yury!

I only saw the discussion here which implied these wouldn't go directly into 
builtins for 3.8 (and I searched here and in GitHub, and couldn't find the PR 
you mentioned either), so I'm curious if that was tracked somewhere. It'd be 
unfortunate if the work I did on that PR couldn't be used, but I'd be even 
happier to have these as builtins.

Thanks again for reviewing when you're back, and have a wonderful vacation in 
the meantime!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34448] incomplete/missing message from configure.ac for usable wchar_t

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8373

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34448] incomplete/missing message from configure.ac for usable wchar_t

2018-08-24 Thread Berker Peksag


Berker Peksag  added the comment:


New changeset 3738fadc670274ecc4649f51b52a93602820a375 by Berker Peksag 
(Michael Osipov) in branch 'master':
bpo-34448: Improve output of usable wchar_t check (GH-8846)
https://github.com/python/cpython/commit/3738fadc670274ecc4649f51b52a93602820a375


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34456] pickle: Missing NULL check in save_global()

2018-08-24 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

We don't bother to fix low probably bugs in 2.7 if the code is too different 
from 3.x. But if you create a PR it will be merged. Seems this should be a 
separate issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34471] _datetime: Missing NULL check in tzinfo_from_isoformat_results()

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset c7f543520161b895796a571881da74e805d0b1eb by Miss Islington (bot) 
in branch '3.7':
closes bpo-34471: _datetime: Add missing NULL check to 
tzinfo_from_isoformat_results. (GH-8869)
https://github.com/python/cpython/commit/c7f543520161b895796a571881da74e805d0b1eb


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34471] _datetime: Missing NULL check in tzinfo_from_isoformat_results()

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8372

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34471] _datetime: Missing NULL check in tzinfo_from_isoformat_results()

2018-08-24 Thread Benjamin Peterson


Benjamin Peterson  added the comment:


New changeset 498845368ff0f6238750ab1d443e7cf4ec98ccd2 by Benjamin Peterson 
(Alexey Izbyshev) in branch 'master':
closes bpo-34471: _datetime: Add missing NULL check to 
tzinfo_from_isoformat_results. (GH-8869)
https://github.com/python/cpython/commit/498845368ff0f6238750ab1d443e7cf4ec98ccd2


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31861] add aiter() and anext() functions to operator module

2018-08-24 Thread Guido van Rossum


Change by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6700] inspect.getsource() returns incorrect source lines at the module level

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset 3e6020c4ddf7acea91efdae770320c6ce06b93db by Miss Islington (bot) 
in branch '3.7':
bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks 
(GH-8864)
https://github.com/python/cpython/commit/3e6020c4ddf7acea91efdae770320c6ce06b93db


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34487] enum _sunder_ names mix metaclass and enum class attributes

2018-08-24 Thread Gerrit Holl

New submission from Gerrit Holl :

In the [`enum` 
module](https://docs.python.org/3/library/enum.html#supported-sunder-names) 
documentation, some of the `_sunder_` names are on `EnumMeta`, whereas others 
are on the produced `Enum` class:

> 8.13.15.3.2. Supported `_sunder_` names

> * `_name_` – name of the member
> * `_value_` – value of the member; can be set / modified in __new__
> * `_missing_` – a lookup function used when a value is not found; may be 
> overridden
> * `_ignore_` – a list of names, either as a list() or a str(), that will not 
> be transformed into members, and will be removed from the final class
> * `_order_` – used in Python 2/3 code to ensure member order is consistent 
> (class attribute, removed during class creation)
> * `_generate_next_value_` – used by the Functional API and by auto to get an 
> appropriate value for an enum member; may be overridden

Experimentally, it appears `_name_` and `_value_` are on the `Enum` class, 
whereas the others are all on the `EnumMeta` class:

In [272]: class Test(enum.Enum): a = 0

In [273]: Test.a._name_
Out[273]: 'a'

In [274]: Test._name
---
AttributeErrorTraceback (most recent call last)
 in ()
> 1 Test._name

/group_workspaces/cems2/fiduceo/Users/gholl/anaconda3/envs/FCDR37a/lib/python3.7/enum.py
 in __getattr__(cls, name)
344 return cls._member_map_[name]
345 except KeyError:
--> 346 raise AttributeError(name) from None
347 
348 def __getitem__(cls, name):

AttributeError: _name

In [275]: Test.a._value_
Out[275]: 0

In [276]: Test._value
---
AttributeErrorTraceback (most recent call last)
 in ()
> 1 Test._value

/group_workspaces/cems2/fiduceo/Users/gholl/anaconda3/envs/FCDR37a/lib/python3.7/enum.py
 in __getattr__(cls, name)
344 return cls._member_map_[name]
345 except KeyError:
--> 346 raise AttributeError(name) from None
347 
348 def __getitem__(cls, name):

AttributeError: _value

In [277]: Test.a._missing_
Out[277]: >

In [278]: Test._missing_
Out[278]: >


This is not clear from the documentation.

--
assignee: docs@python
components: Documentation
messages: 324003
nosy: Gerrit.Holl, docs@python
priority: normal
severity: normal
status: open
title: enum _sunder_ names mix metaclass and enum class attributes
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6700] inspect.getsource() returns incorrect source lines at the module level

2018-08-24 Thread miss-islington


miss-islington  added the comment:


New changeset 0e707b4c6a47086d8e723c7a010c3f5cf8296946 by Miss Islington (bot) 
in branch '3.6':
bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks 
(GH-8864)
https://github.com/python/cpython/commit/0e707b4c6a47086d8e723c7a010c3f5cf8296946


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6700] inspect.getsource() returns incorrect source lines at the module level

2018-08-24 Thread Tal Einat


Change by Tal Einat :


--
pull_requests: +8371

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue33569] dataclasses InitVar does not maintain any type info

2018-08-24 Thread Eric V. Smith


Eric V. Smith  added the comment:

Can you create a pull request? It's easier to review that way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6700] inspect.getsource() returns incorrect source lines at the module level

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8369

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6700] inspect.getsource() returns incorrect source lines at the module level

2018-08-24 Thread miss-islington


Change by miss-islington :


--
pull_requests: +8370

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue6700] inspect.getsource() returns incorrect source lines at the module level

2018-08-24 Thread Tal Einat


Tal Einat  added the comment:


New changeset 91cb298f811961277fd4cc4a32211899d48bedcb by Tal Einat (Vladimir 
Matveev) in branch 'master':
bpo-6700: Fix inspect.getsourcelines for module level frames/tracebacks 
(GH-8864)
https://github.com/python/cpython/commit/91cb298f811961277fd4cc4a32211899d48bedcb


--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34486] "RuntimeError: release unlocked lock" when starting a thread

2018-08-24 Thread Uosiu

Stanisław Skonieczny (Uosiu)  added the comment:

This scripts runs start_threads.py and send a signal to it. After some number 
of loops problem can be reproduced.

--
Added file: https://bugs.python.org/file47762/test.sh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34486] "RuntimeError: release unlocked lock" when starting a thread

2018-08-24 Thread Uosiu

New submission from Stanisław Skonieczny (Uosiu) 
:

Sometimes when thread is starting it raises "RuntimeError: release unlocked 
lock". That is when signal handler is invoked in the same time.

Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 551, in wait
signaled = self._cond.wait(timeout)
  File "/usr/lib/python3.6/threading.py", line 304, in wait
self._acquire_restore(saved_state)
  File "start_threads.py", line 12, in sighandler
raise MyException("got signal")
__main__.MyException: got signal

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "start_threads.py", line 28, in 
thread.start()
  File "/usr/lib/python3.6/threading.py", line 851, in start
self._started.wait()
  File "/usr/lib/python3.6/threading.py", line 552, in wait
return signaled
  File "/usr/lib/python3.6/threading.py", line 243, in __exit__
return self._lock.__exit__(*args)
RuntimeError: release unlocked lock

I have attached to files that reproduce the problem. First runs python and 
starts noop threads in the loop. It uses signal handler that raises exception. 
Second is a bash script to run python script and send signal to it. After some 
time, when signal is handled in unfortunate place, python script breaks due to 
unreleased lock.

--
components: Library (Lib)
files: start_threads.py
messages: 323998
nosy: Stanisław Skonieczny (Uosiu)
priority: normal
severity: normal
status: open
title: "RuntimeError: release unlocked lock" when starting a thread
versions: Python 3.6
Added file: https://bugs.python.org/file47761/start_threads.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31861] add aiter() and anext() functions to operator module

2018-08-24 Thread Yury Selivanov


Yury Selivanov  added the comment:

Great, thanks! There's someone working on a pr to add them as builtins (so that 
pr is in C). I'll have to take a look when i'm back from vacation at that pr 
too, before we can make a decision. 

I've been to europython, pycon ru, and pybay conferences recently and the 
number of people who want these as builtins was surprisingly high.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34347] AIX: test_utf8_mode.test_cmd_line fails

2018-08-24 Thread Michael Felt


Michael Felt  added the comment:

On 23/08/2018 19:14, Michael Felt wrote:
> Michael Felt  added the comment:
>
> On 23/08/2018 12:51, STINNER Victor wrote:
>> STINNER Victor  added the comment:
>>
>> Your issue is about decoding command line argument which is done from main() 
>> function. It doesn't use Python codecs, but functions like Py_DecodeLocale().
> This is beyond my understanding atm.
> Early on I tried making the expected just be 'arg' and went from
> situation A to situation B - which looked much closer, BUT, the 'types'
> differed:
>
> Situaltion A (original)
> AssertionError: "['h\\xc3\\xa9\\xe2\\x82\\xac']" != 
> "['h\\udcc3\\udca9\\udce2\\udc82\\udcac']"
> - ['h\xc3\xa9\xe2\x82\xac']
> + ['h\udcc3\udca9\udce2\udc82\udcac']
>  : ISO8859-1:['h\xc3\xa9\xe2\x82\xac']
>
> I tried saying the "expected" is arg, but arg is still a byte object, the 
> cmd_line result is not (printed as such).
>
> Situation B
> AssertionError: "['h\\xc3\\xa9\\xe2\\x82\\xac']" != 
> "[b'h\\xc3\\xa9\\xe2\\x82\\xac']"
> - ['h\xc3\xa9\xe2\x82\xac']
> + [b'h\xc3\xa9\xe2\x82\xac']
> ?  +
>  : ISO8859-1:['h\xc3\xa9\xe2\x82\xac']
>
> After further digging - to understand why it was coming as "\x encoding 
> rather than \udc"
>
> I looked at what was happening here:
>
> out = self.get_output('-X', utf8_opt, '-c', code, arg, **kw)
> becomes
> out = self.get_output('-X', utf8_opt, '-c', code, 
> 'h\xe9\u20ac'.encode('utf-8'), **kw)
> becomes
> out = self.get_output('-X', utf8_opt, '-c', code, b'h\xc3\xa9\xe2\x82\xac', 
> **kw)
>
> And finally, at the CLI becomes:
> ['/data/prj/python/python3-3.8/python', '-X', 'faulthandler', '-X', 'utf8', 
> '-c', 'import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), 
> ascii(sys.argv[1:])))', b'h\xc3\xa9\xe2\x82\xac']
>
> /data/prj/python/python3-3.8/python '-X' 'faulthandler' '-X' 'utf8' '-c' 
> 'import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), 
> ascii(sys.ar
> gv[1:])))', b'h\xc3\xa9\xe2\x82\xac'
> UTF-8:['bh\\xc3\\xa9\\xe2\\x82\\xac']
>
> /data/prj/python/python3-3.8/python '-X' 'faulthandler' '-X' 'utf8=0' '-c' 
> 'import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), 
> ascii(sys.
> argv[1:])))', b'h\xc3\xa9\xe2\x82\xac'
> ISO8859-1:['bh\\xc3\\xa9\\xe2\\x82\\xac']
>
> Note:
> /data/prj/python/python3-3.8/python '-X' 'faulthandler' '-X' 'utf8=0' '-c' 
> 'import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), 
> ascii(sys.
> argv[1:])))', 'h\udcc3\udca9\udce2\udc82\udcac'
> ISO8859-1:['h\\udcc3\\udca9\\udce2\\udc82\\udcac']
>
> /data/prj/python/python3-3.8/python '-X' 'faulthandler' '-X' 'utf8=0' '-c' 
> 'import locale, sys; print("%s:%s" % (locale.getpreferredencoding(), 
> ascii(sys.
> argv[1:])))', b'h\udcc3\udca9\udce2\udc82\udcac'
> ISO8859-1:['bh\\udcc3\\udca9\\udce2\\udc82\\udcac']
>
> root@x066:[/data/prj/python/python3-3.8]/data/prj/python/python3-3.8/python 
> '-X' 'faulthandler' '-X' 'utf8' '-c' 'import locale, sys; print("%s:%s" % (>
> UTF-8:['bh\\udcc3\\udca9\\udce2\\udc82\\udcac']
>
> Summary:
> a) concerned about how b'h' becomes 'bh'
> b) whatwever argv[1] is, is very close to what is returned - so whatever 
> happens durinf the transformation from 
> self.get_output('-X', utf8_opt, '-c', code, arg, **kw)
>  determines the output and the (failed) comparison.
p.s. also tried:
michael@x071:[/data/prj/python/git/python3-3.8]/data/prj/python/python3-3.8/python
'-X' 'faulthandler' '-X' 'utf8=0' '-c' 'import locale, sys;
print("%s:%s" % (locale.getpreferredencoding(), ascii(sys.argv[1:])))',
'h\xe9\u20ac'.encode\('utf-8'\)
ISO8859-1:['h\\xe9\\u20ac.encode(utf-8)']
michael@x071:[/data/prj/python/git/python3-3.8]/data/prj/python/python3-3.8/python
'-X' 'faulthandler' '-X' 'utf8=1' '-c' 'import locale, sys;
print("%s:%s" % (locale.getpreferredencoding(), ascii(sys.argv[1:])))',
'h\xe9\u20ac'.encode\('utf-8'\)
UTF-8:['h\\xe9\\u20ac.encode(utf-8)']

Really unclear to me what this test is trying to verify. The CLI seems
to just 'echo' what it is provided.
>>> Question 1: why is windows excluded? Because it does not use UTF-8 as it's 
>>> default (it's default is CP1252)
>> Windows uses wmain() which gets command line arguments as wchar_t* strings: 
>> Unicode. No decoding is needed.
>>
>> --
>>
>> ___
>> Python tracker 
>> 
>> ___
>>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com