date:20170106


Changes by Berker Peksag :


--
nosy: +berker.peksag
stage:  -> patch review
type:  -> behavior
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29145] failing overflow checks in replace_*


Serhiy Storchaka added the comment:

The code in PyUnicode_Join() checks on overflow after making an addition.

sz += PyUnicode_GET_LENGTH(item);
item_maxchar = PyUnicode_MAX_CHAR_VALUE(item);
maxchar = Py_MAX(maxchar, item_maxchar);
if (i != 0)
sz += seplen;
if (sz < old_sz || sz > PY_SSIZE_T_MAX) {
PyErr_SetString(PyExc_OverflowError,
"join() result is too long for a Python string");
goto onError;
}

Maybe there are other cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29186] TimeoutError isn't being raised?

2017-01-06 Thread Xiang Zhang


Changes by Xiang Zhang :


--
stage:  -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29187] Pickle failure is raising AttributeError and not PicklingError


Serhiy Storchaka added the comment:

Python implementation of pickle still raises PicklingError. Seems this was not 
intentional change.

>>> pickle._dumps(func()())
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 918, in save_global
obj2, parent = _getattribute(module, name)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 266, in _getattribute
.format(name, obj))
AttributeError: Can't get local attribute 'func..C' on 

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 1544, in _dumps
_Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 409, in dump
self.save(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 521, in save
self.save_reduce(obj=obj, *rv)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 605, in save_reduce
save(cls)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 978, in save_type
return self.save_global(obj)
  File "/home/serhiy/py/cpython/Lib/pickle.py", line 922, in save_global
(obj, module_name, name))
_pickle.PicklingError: Can't pickle .C'>: it's 
not found as __main__.func..C

--
components: +Extension Modules -Library (Lib)
nosy: +alexandre.vassalotti, serhiy.storchaka
stage:  -> patch review
type:  -> behavior
versions: +Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29190] Avoid possible errors in comparing strings


New submission from Serhiy Storchaka:

PyUnicode_Compare() and PyUnicode_RichCompare() can raise an exception if one 
of arguments is not ready unicode object. The result is not always checked for 
error. Proposed patch gets rid of possible bugs. PyUnicode_Compare() and 
PyUnicode_RichCompare() in Modules/_pickle.c are replaced with 
_PyUnicode_EqualToASCIIString() and _PyUnicode_EqualToASCIIId() which never 
fail. Additional check is added in Modules/_decimal/_decimal.c to ensure that 
the string which is came from a user code is ready.

All other occurrences of PyUnicode_Compare() seems are called only with ready 
unicode objects.

--
components: Extension Modules
files: unicode_compare.patch
keywords: patch
messages: 284895
nosy: serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Avoid possible errors in comparing strings
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46184/unicode_compare.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29133] Minor inaccuracy in shlex.shlex punctuation_chars example


Berker Peksag added the comment:

I'd probably write it without the for loop:

text = "a && b; c && d || e; f >'abc'; (def \"ghi\")"

result = shlex.shlex(text)
print(f"Old behavior: {list(result)}")

result = shlex.shlex(text, punctuation_chars=True)
print(f"New behavior: {list(result)}")

Or just:

>>> import shlex
>>> text = "a && b; c && d || e; f >'abc'; (def \"ghi\")"
>>> list(shlex.shlex(text))
['a', '&', '&', 'b', ';', 'c', '&', '&', 'd', '|', '|', 'e', ';', 'f', '>', 
"'abc'", ';', '(', 'def', '"ghi"', ')']
>>> list(shlex.shlex(text, punctuation_chars=True))
['a', '&&', 'b', ';', 'c', '&&', 'd', '||', 'e', ';', 'f', '>', "'abc'", 
';', '(', 'def', '"ghi"', ')']

(Adding Vinay to nosy list to get his feedback since he wrote the original 
example.)

--
nosy: +berker.peksag, vinay.sajip
stage:  -> patch review
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16026] csv.DictReader argument names documented incorrectly


Roundup Robot added the comment:

New changeset 198edd926751 by Berker Peksag in branch '3.6':
Issue #16026: Fix parameter names of DictReader and DictWriter
https://hg.python.org/cpython/rev/198edd926751

New changeset 63c5531cfdf7 by Berker Peksag in branch 'default':
Issue #16026: Merge from 3.6
https://hg.python.org/cpython/rev/63c5531cfdf7

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16026] csv.DictReader argument names documented incorrectly


Berker Peksag added the comment:

Thanks for the patches James and Greg!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29189] Broken indentation in FancyURLopener documentation


Berker Peksag added the comment:

Thanks for the review, Senthil.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29189] Broken indentation in FancyURLopener documentation


Roundup Robot added the comment:

New changeset 31172ecb9e40 by Berker Peksag in branch '2.7':
Issue #29189: Fix broken indentation in FancyURLopener documentation
https://hg.python.org/cpython/rev/31172ecb9e40

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29189] Broken indentation in FancyURLopener documentation

2017-01-06 Thread Senthil Kumaran


Senthil Kumaran added the comment:

The patch looks good to me, and can committed. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29189] Broken indentation in FancyURLopener documentation


New submission from Berker Peksag:

I noticed this while taking a look at urllib docs for issue 29182. Indentation 
of prompt_user_passwd() is broken and it looks like it's part of the note 
directive: 
https://docs.python.org/2/library/urllib.html#urllib.FancyURLopener.prompt_user_passwd

Python 3 version renders fine: 
https://docs.python.org/3/library/urllib.request.html#urllib.request.FancyURLopener.prompt_user_passwd

Here is a patch.

--
assignee: docs@python
components: Documentation
files: docfix.diff
keywords: patch
messages: 284888
nosy: berker.peksag, docs@python, orsenthil
priority: normal
severity: normal
stage: patch review
status: open
title: Broken indentation in FancyURLopener documentation
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file46183/docfix.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29182] Remove the warning in urllib docs that it doesn't do certificate validate by default.


Changes by Berker Peksag :


--
stage: patch review -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28180] sys.getfilesystemencoding() should default to utf-8

2017-01-06 Thread INADA Naoki


INADA Naoki added the comment:

>> stderr is used to log errors. Getting a new error when trying to log
>> an error is kind of annoying.
>
> Hm, what bad surprise/error could appear that would not appear with 
> backslashescape?

$ cat badfilename.py 
badfn = "こんにちは".encode('euc-jp').decode('utf-8', 'surrogateescape')
print("bad filename:", badfn)

$ PYTHONIOENCODING=utf-8:backslashreplace python3 badfilename.py 
bad filename: \udca4\udcb3\udca4\udcf3\udca4ˤ\udcc1\udca4\udccf

$ PYTHONIOENCODING=utf-8:surrogateescape python3 badfilename.py 
bad filename: �ˤ���

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28180] sys.getfilesystemencoding() should default to utf-8


Sworddragon added the comment:

> What do you mean by "make the C locale"?

I was pointing to the Platform Support Changes of PEP 538.


> I'm not sure of the name of each mode yet.
>
> After having written the "Use Cases" section and especially the
> Mojibake column of results, I consider the option of renaming the
> "UTF-8 mode" to "YOLO mode".

Assumingly YOLO is meant to be negative: Things are whirling in my mind. 
Eventually you want to save your joker :>


> Using surrogateescape means that you pass through undecodable bytes
> from inputs to stderr which can cause various kinds of bad surprises.
>
> stderr is used to log errors. Getting a new error when trying to log
> an error is kind of annoying.

Hm, what bad surprise/error could appear that would not appear with 
backslashescape?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28961] unittest.mock._Call ignores `name` parameter


Berker Peksag added the comment:

IIRC 3.5.3rc1 is already tagged so the 3.5 branch is open for 3.5.4. Anything 
that needs to be in 3.5.3 should be cherry-picked by the RM at this point if 
I'm not mistaken (and note that I don't have time check, but 3.5.3 may be the 
last bugfix release of 3.5 series so you may just skip 3.5 :))

Like I already said in msg284589, the test code doesn't follow the current 
style in Lib/unittest/test/testmock/testhelpers.py and it can be simplified 
like the following patch:

 def test_call_with_name(self):
-self.assertEqual(
-'foo',
-_Call((), 'foo')[0],
-)
-self.assertEqual(
-'',
-_Call((('bar', 'barz'), ), )[0]
-)
-self.assertEqual(
-'',
-_Call((('bar', 'barz'), {'hello': 'world'}), )[0]
-)
+self.assertEqual(_Call((), 'foo')[0], 'foo')
+self.assertEqual(_Call((('bar', 'barz'),),)[0], '')
+self.assertEqual(_Call((('bar', 'barz'), {'hello': 'world'}),)[0], '')

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28180] sys.getfilesystemencoding() should default to utf-8


STINNER Victor added the comment:

Sworddragon added the comment:
> (for me and maybe others that is explicitly preferred but maybe this depends 
> on each individual)

That's why the PEP 540 has options to enable to disable its UTF-8 mode(s).

> If I'm not wrong PEP 538 improves this for the output too but input handling 
> will still suffer from the overall issue while PEP 540 does also solve this 
> case.

The PEP 538 works fine if all inputs and outputs are encoded to UTF-8.
I understand that it's a deliberate choice to fail on
decoding/encoding error (to not use surrogateescape), but I can be
wrong.

> Also PEP 540 would not make the C locale and thus eventually some systems 
> potentially unsupported (but it might be an acceptable trade-off if we should 
> really go PEP 538).

What do you mean by "make the C locale"?

> Specific for PEP 540:
>
>> The POSIX locale enables the UTF-8 mode
>
> Non-strict I assume?

Yes, non strict.

I'm not sure of the name of each mode yet.

After having written the "Use Cases" section and especially the
Mojibake column of results, I consider the option of renaming the
"UTF-8 mode" to "YOLO mode".

>> UTF-8 /backslashreplace
>
> Was/is the reason to use backslashreplace for sys.stderr to guarantee that 
> the developer/user sees the error messages?

Yes.

> Might it make sense to also use surrogateescape instead of backslashescape 
> for sys.stderr in UTF-8 non-strict mode to be consistent here?

Using surrogateescape means that you pass through undecodable bytes
from inputs to stderr which can cause various kinds of bad surprises.

stderr is used to log errors. Getting a new error when trying to log
an error is kind of annoying.

Victor

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29174] 'NoneType' object is not callable in subprocess.py


STINNER Victor added the comment:

Martin Panter:
> Victor opened Issue 27068 about adding a Popen.detach() method, which such 
> code could use to opt out of the warning.

I opened the issue because you asked me to open it, but I'm not
convinced yet that the design would work. I don't understand yet who
is responsible of the pipes for example, especially pipes opened by
the Popen object itself (ex: stdout=PIPE), not passed to Popen
constructor. It's not as simple as getting a file descriptor as
file.detach() or socket.detach(), a Popen object is made of multiple
resources (pid and pipes at least).

> 2. Revert the warning, and in a future release (e.g. 3.7), add it back along 
> with a way to opt out of the warning.

For this specific issue, the ResourceWarning is correct. I don't
understand the use case of explicitly turning this warning off on this
specific example?

If your output is flooded by ResourceWarning warnings, it's easy to
configure Python to ignore them. Example, simplest option: python3
-Wignore script.py. But you are only going to hide a real issue in
your code. ResourceWarning exists to help you to reduce your resource
consumption.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28180] sys.getfilesystemencoding() should default to utf-8


Sworddragon added the comment:

On looking into PEP 538 and PEP 540 I think PEP 540 is the way to go. It 
provides an option for a stronger encapsulation for the de-/encoding logic 
between the interpreter and the developer. Instead of caring about error 
handling the developer has now to care about mojibake handling (for me and 
maybe others that is explicitly preferred but maybe this depends on each 
individual). If I'm not wrong PEP 538 improves this for the output too but 
input handling will still suffer from the overall issue while PEP 540 does also 
solve this case. Also PEP 540 would not make the C locale and thus eventually 
some systems potentially unsupported (but it might be an acceptable trade-off 
if we should really go PEP 538).


Specific for PEP 540:

> The POSIX locale enables the UTF-8 mode

Non-strict I assume?


> UTF-8 /backslashreplace

Was/is the reason to use backslashreplace for sys.stderr to guarantee that the 
developer/user sees the error messages? Might it make sense to also use 
surrogateescape instead of backslashescape for sys.stderr in UTF-8 non-strict 
mode to be consistent here?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29174] 'NoneType' object is not callable in subprocess.py

2017-01-06 Thread Martin Panter


Martin Panter added the comment:

The ResourceWarning was added by Issue 26741.

I agree that there are legitimate reasons why pre-3.6 code may avoid calling 
Popen.wait() and equivalent. Victor opened Issue 27068 about adding a 
Popen.detach() method, which such code could use to opt out of the warning.

I don’t think there should be a special exemption for the warning at shutdown 
time. I think we should either:

1. Accept that you should never destroy a 3.6 Popen object without first 
“waiting” on its child (or zombie), or:

2. Revert the warning, and in a future release (e.g. 3.7), add it back along 
with a way to opt out of the warning.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29188] Backport random.c from Python 3.5 to Python 2.7


STINNER Victor added the comment:

> I think it is far too late to be making these kind of changes to 2.7.

I would prefer to use the "same code" (or almost) on all maintained versions of 
Python: 2.7, 3.5, 3.6 and 3.7. It should ease the maintenance for bugfixes and 
enhancements.

It seems like we want to backport security enhancements from Python 3 to Python 
2.7: see the PEP 466. Copying random.c from Python 3 would add support for 
getrandom() which is nice to have since it avoids a private file descriptor 
(which causes many issues, even if the most important issues are already worked 
around in Python 2.7 using fstat()).

The minimum required change on Python 2.7 is to not use getentropy() on Linux 
to support the glibc 2.24: see attached getentropy_linux.patch if you don't 
want the backport.

--
keywords: +patch
Added file: http://bugs.python.org/file46182/getentropy_linux.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27632] build on AIX fails when builddir != srcdir, more than bad path to ld_so_aix

2017-01-06 Thread Martin Panter


Martin Panter added the comment:

Regarding reopening Issue 10656, whatever you think is more appropriate. You 
just have to judge whether it is the same use case, the same code affected, etc.

Issue 16189 and Issue 25825 were about updating to match recent changes to 
directory names, and I thought we decided the changes were not applicable to 
2.7.

Regarding LDSHARED vs BLDSHARED, isn’t this the same as Issue 28311, which lead 
to Issue 18235? If you can try the patch I mentioned at 
, I suspect it may help. Let me 
know if you need help adapting the patch for 2.7.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29179] Py_UNUSED is not documented


Changes by Raymond Hettinger :


--
assignee: docs@python -> larry
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29188] Backport random.c from Python 3.5 to Python 2.7


Raymond Hettinger added the comment:

I think it is far too late to be making these kind of changes to 2.7.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29023] Results of random.seed() call with integer argument should be claimed deterministic.


Changes by Raymond Hettinger :


--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29023] Results of random.seed() call with integer argument should be claimed deterministic.


Roundup Robot added the comment:

New changeset 7d6ebd206cd6 by Raymond Hettinger in branch '2.7':
Issue #29023:  Clarify that ints and longs are always deterministic seeds for 
random.
https://hg.python.org/cpython/rev/7d6ebd206cd6

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29186] TimeoutError isn't being raised?

2017-01-06 Thread YoSTEALTH


Changes by YoSTEALTH :


--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29157] random.c: Prefer getrandom() over getentropy() to support glibc 2.24 on Linux


Changes by STINNER Victor :


--
title: random.c: Prefer getrandom() over getentropy(), handle ENOSYS in 
py_getentropy() -> random.c: Prefer getrandom() over getentropy() to support 
glibc 2.24 on Linux

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29188] Backport random.c from Python 3.5 to Python 2.7


New submission from STINNER Victor:

Python 3.6 uses the new getrandom() function/syscall on Linux and Solaris to 
get random bytes with no file descriptor: it prevents EMFILE and ENFILE errors 
or surprises when opening a first file (and looking at its file descriptor).

I propose to copy and adapt Python/random.c from Python 3.5 when Python 3.5 
will be updated for the issue #29157. Python 2.7 requires extra changes:

* configure.ac: need to check linux/random.h in AC_CHECK_HEADERS()
* random.c: need to keep vms_urandom() function (Python 2.7 still supports VMS!)
* Python 2.7 doesn't implement the PEP 475 (EINTR) and so don't have functions 
like _Py_read() which handles EINTR for us.

See also the issue #29157 for the latest change in random.c: prefer getrandom() 
over getentropy() to support the glibc 2.24.

--
messages: 284876
nosy: haypo
priority: normal
severity: normal
status: open
title: Backport random.c from Python 3.5 to Python 2.7
type: security
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale


STINNER Victor added the comment:

> But maybe I'm just missing something.

This issue fixed exactly one use case: "List a directory into stdout" (similar 
to the UNIX "ls" or Windows "dir" commands):
https://www.python.org/dev/peps/pep-0540/#list-a-directory-into-stdout

Your use case is more "Display Unicode characters into stdout":
https://www.python.org/dev/peps/pep-0540/#display-unicode-characters-into-stdout

This use case is not supported by the issue. It should be fixed by PEP 538 or 
PEP 540.

Please join the happy discussion on the python-ideas mailing list to discuss 
how to "force UTF-8": this issue is closed, you shouldn't add new comments 
(other people will not see your comments).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29030] argparse: choices override metavar

2017-01-06 Thread paul j3


paul j3 added the comment:

subparsers is an example of choices displaying as the metavar.  From the 
documentation example:

usage: PROG [-h] [--foo] {a,b} ...

positional arguments:
  {a,b}   sub-command help

-

To the main parser, the 'subparser' is a positional argument, with an optional 
'dest' parameter (default is SUPPRESS), and {a,b} are the choices derived from 
the define subparser names.

The 'title' and 'description' parameters are used to create an Argument_group.

So any attempt to change how the metavar is created from choices has the 
potential of changing the subparser display.

That does suggest another formatting option - put your positional argument 
(with choices) in a custom Argument_Group.

In [658]: p=argparse.ArgumentParser()
In [659]: g=p.add_argument_group(title='lengths')
In [660]: g.add_argument('length',choices=[1,2,3]);
In [661]: p.print_help()
usage: ipython3 [-h] {1,2,3}

optional arguments:
  -h, --help  show this help message and exit

lengths:
  {1,2,3}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale


Sworddragon added the comment:

The point is this ticket claims to be using the surrogateescape error handler 
for sys.stdout and sys.stdin for the C locale. I have never used 
surrogateescape explicitly before and thus have no experience for it and 
consulting the documentation mentions throwing an exception only for the strict 
error handler. I don't see anything that would make me think that 
surrogateescape would throw here an exception too. But maybe I'm just missing 
something.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29157] random.c: Prefer getrandom() over getentropy(), handle ENOSYS in py_getentropy()


STINNER Victor added the comment:

random-py35.patch: Patch for the 3.5 branch. My prepared commit message:
---
Issue #29157: Prefer getrandom() over getentropy()

Copy and then adapt Python/random.c from default branch. Difference between 3.5
and default branches:

* Python 3.5 only uses getrandom() in non-blocking mode: flags=GRND_NONBLOCK
* If getrandom() fails with EAGAIN: py_getrandom() immediately fails and
  remembers that getrandom() doesn't work.
* Python 3.5 has no _PyOS_URandomNonblock() function: _PyOS_URandom()
  works in non-blocking mode on Python 3.5
---

It seems like Python 3.5 is close to a release, I prefer to wait after the 
release to fix this issue. I don't think that many Linux distributions are 
affected, since the issue only occurs with glibc 2.24 which is very recent.

@Larry: Do you want this change in Python 3.5.3? The change is quite large.

--
nosy: +larry
Added file: http://bugs.python.org/file46181/random-py35.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29157] random.c: Prefer getrandom() over getentropy(), handle ENOSYS in py_getentropy()


STINNER Victor added the comment:

Christian Heimes: "I'm doing a review now."

Follow-up on #python-dev (IRC):

 haypo: yes, I looked at the patch and did not see any obvious problem 
with it. Didn't I tell you?
 haypo: maybe I forgot :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29157] random.c: Prefer getrandom() over getentropy(), handle ENOSYS in py_getentropy()


Roundup Robot added the comment:

New changeset f8e24a0a1124 by Victor Stinner in branch '3.6':
Issue #29157: Prefer getrandom() over getentropy()
https://hg.python.org/cpython/rev/f8e24a0a1124

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29187] Pickle failure is raising AttributeError and not PicklingError

2017-01-06 Thread Matt Dodge


New submission from Matt Dodge:

When failing to pickle something (like a locally scoped class) the 
documentation indicates that a PicklingError should be raised.
Doc links:
 - 
https://docs.python.org/3/library/pickle.html?highlight=pickle#pickle-picklable
 - 
https://docs.python.org/3.5/library/pickle.html#what-can-be-pickled-and-unpickled

However, instead I'm seeing AttributeError get raised instead, starting in 
Python 3.5. In Python 3.4 PicklingErrror was raised, as expected.

To reproduce, use the following file 
def func():
class C: pass
return C
import pickle
pickle.dumps(func()())

In Python 3.4 you see:
Traceback (most recent call last):
  File "pickletest.py", line 5, in 
pickle.dumps(func()())
_pickle.PicklingError: Can't pickle .C'>: 
attribute lookup C on __main__ failed

But in 3.5/3.6 you see:
Traceback (most recent call last):
  File "pickletest.py", line 5, in 
pickle.dumps(func()())
AttributeError: Can't pickle local object 'func..C'

I don't necessarily mind that a different exception is being raised, but how 
should we be handling exceptions while pickling? Catch all exceptions? That 
doesn't feel right to me, but if we're trying to pickle data out of our control 
I'm not sure what else to do.

FYI, the UnpicklingError documentation 
(https://docs.python.org/3/library/pickle.html?highlight=pickle#pickle.UnpicklingError)
 indicates that other exceptions can possibly be raised during unpickling. I 
assume that is more related to the fact that the pickled data may not fit into 
the current class/module structure though, so I think it's unrelated.

--
components: Library (Lib)
messages: 284869
nosy: Matt.Dodge
priority: normal
severity: normal
status: open
title: Pickle failure is raising AttributeError and not PicklingError
versions: Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29186] TimeoutError isn't being raised?

2017-01-06 Thread YoSTEALTH


New submission from YoSTEALTH:

TimeoutError isn't being raised?

My Python Version: 3.5.1 (64bit, linux)

# Document:
https://docs.python.org/3/library/exceptions.html#TimeoutError
""" exception TimeoutError
Raised when a system function timed out at the system level. Corresponds to 
errno ETIMEDOUT.
New in version 3.3: All the above OSError subclasses were added.
See also PEP 3151 - Reworking the OS and IO exception hierarchy """

# PEP: According to pep-3151
link: https://www.python.org/dev/peps/pep-3151/
""" TimeoutError : connection timed out (ETIMEDOUT); this can be re-cast as a 
generic timeout exception, replacing socket.timeout and also useful for other 
types of timeout (for example in Lock.acquire()) """


# This Does NOT Work:
def Send(conn, data):
# Set Timeout.
conn.settimeout(3.0)
try:
while data:
sent = conn.send(data)
data = data[sent:]
except TimeoutError as e:
print("TimeoutError:", e)  #
close_connection()
else:
pass  # Do Stuff...


# This Works
def Send(conn, data):
# Set Timeout.
conn.settimeout(3.0)
try:
while data:
sent = conn.send(data)
data = data[sent:]
except socket.timeout as e:
print("socket.timeout:", e)  # socket.timeout: timed out
close_connection()
else:
pass  # Do Stuff...


# This Works
def Send(conn, data):
# Set Timeout.
conn.settimeout(3.0)
try:
while data:
sent = conn.send(data)
data = data[sent:]
except OSError as e:
print('ERROR Send:', e)  # ERROR Send: timed out
close_connection()
else:
pass  # Do Stuff...


According to PEP "TimeoutError" is suppose to replace "socket.timeout" but it 
doesn't seem to work! Any ideas why?

--
messages: 284868
nosy: YoSTEALTH
priority: normal
severity: normal
status: open
title: TimeoutError isn't being raised?
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29174] 'NoneType' object is not callable in subprocess.py

2017-01-06 Thread ita1024


ita1024 added the comment:

The point #3 was referring to the new requirement for an atexit handler in 
order to not only kill the processes but to also wait for them at interpreter 
shutdown. The sub-processes (and associated resources) in the example are 
definitely freed as the parent process is terminating.

The recommended handler is not even always desirable (spawning daemon 
processes, key agents), it increases the code verbosity, impacts performance, 
and can even cause problems as child processes cannot always be waited on 
reliably (python 2 but also child -D state and platform-specific restrictions).

I suggest to disable such warnings during interpreter shutdown.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue

2017-01-06 Thread Brett Cannon


Changes by Brett Cannon :


--
pull_requests: +10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista


STINNER Victor added the comment:

I dislike adding a lpAttributeList attribute: it's too close to the exact 
implementation of Windows may change in the future. I would prefer a more high 
level API.

Since the only known use case today is to pass handles, I propose to focus on 
this use case: add a new pass_handles parameter to Popen, similar to pass_fds.

I see that your patch is able to set close_fds to True on Windows: great job! 
It would be a great achievement to finally fix this last known race condition 
of subprocess on Windows!

So thank you for working on this!


> As for pass_fds: as you noted, it has it's own share of complexities and 
> issues and I think it's best to leave it to a separate patch/issue.

pass_fds would be "nice to have", but I prefer to stick first to the native and 
well supported handles on Windows. For me, using file descriptors on Windows is 
more a "hack" to be able to write code working on Windows and UNIX, but since 
it's not natively supported on Windows, it comes with own set of issues.

IMHO it's better supported to work on handles.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista


STINNER Victor added the comment:

Python already has a multiprocessing module which is able to pass handles 
(maybe also FD? I don't know) to child processes on Windows. I found some code 
in Lib/multiprocessing/reduction.py:
- duplicate()
- steal_handle()
- send_handle()

But the design doesn't really fit the subprocess module, since this design 
requires that the child process communicates with the parent process. On UNIX, 
fork()+exec() is used, so we can execute a few instructions after fork, which 
allows to pass an exception from the child to the parent. On Windows, 
CreateProcess() is used which doesn't allow directly to execute code before 
running the final child process.

The PEP 446 describes a solution using a wrapper process, so 
parent+wrapper+child, 3 processes. IMHO the best design for subprocess is 
really PROC_THREAD_ATTRIBUTE_HANDLE_LIST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29178] Adding bytes.frombuffer(byteslike) constructor


Serhiy Storchaka added the comment:

> Which virtually no one follows :(

Sad. But adding bytes.frombuffer() wouldn't make it magically used. If you are 
aware of the problem, you can use the above two-liner as well as 
bytes.frombuffer(). You even can use it in the current code with older Python 
releases, unlike to bytes.frombuffer() which would need 3.7.

> Any protocol parsing code has a lot of slicing.

How much code you expect to update with bytes.frombuffer()? And why not use the 
above two-liner instead?

> > There is also a problem with returned type for subclasses (this is always
> > a problem for alternate constructors).
> Good point. How do we usually solve this in CPython?

It is deemed that returning an instance of a subclass is more preferable. 
Otherwise you could use separate function rather of a class method. But it is 
not easy. You need either pass a memoryview or bytes instance to class 
constructor, or (only for mutable arrays) create an empty instance and 
concatenate a buffer to it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale


STINNER Victor added the comment:

"I thought with the surrogateescape error handler now being used for sys.stdout 
this would not throw an exception but I'm getting this: (...)"

Please see the two recently proposed PEP: Nick's PEP 538 and my PEP 540, both 
propose (two different) solutions to your issue, especially for the POSIX 
locale (aka "C" locale).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29116] Make str and bytes error messages on concatenation conform with other sequences


Serhiy Storchaka added the comment:

You can concatenate any object supporting the buffer protocol to bytes and 
bytearray.

Current error message for bytes looks awkward too, because it says "bytes to 
othertype" instead of "othertype to bytes".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests of test_logging when bind() raises PermissionError (non-root user on Android)


Vinay Sajip added the comment:

> including the SysLogHandlerTest, which wasn't reported

Sorry, I appear to have lost the ability to read :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix


Serhiy Storchaka added the comment:

Yes this prevents the injection.

The injection is possible because the patch is substituted in the string 
without any escaping. Your fix is not enough. The real path to a Tix 
installation can contain special characters: '\', '{' or '}'.

My patch first sets a path to a Tcl variable (there is no an injection, because 
special API is used instead of evaluating a generated script), and then use 
this variable in the script (unlike to Unix shell Tcl doesn't reparse the 
command after substituting variables).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29117] dir() should include dunder attributes of the unbound method

2017-01-06 Thread Terry J. Reedy


Terry J. Reedy added the comment:

I tested your last claim and it is true as far as I went.

>>> C.f.__annotations__
{'a': }
>>> C().f.__annotations__
{'a': }
>>> C.f.__code__
", line 2>
>>> C().f.__code__
", line 2>

--
nosy: +terry.reedy
stage:  -> test needed
type:  -> behavior
versions: +Python 3.6, Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29116] Make str and bytes error messages on concatenation conform with other sequences

2017-01-06 Thread Terry J. Reedy


Terry J. Reedy added the comment:

By default, error message wording changes are 'enhancements' that wait for the 
next x.y.0 release unless the current wording is positively wrong'.  This is 
different from doc changes because there are tests depending on error messages. 
'Inconsistent' or 'awkward' is not the same as 'wrong'.

I do agree that the clipped "must be str, not bytes' is awkward.  What (which) 
is it that must be str?  A fleshed out and positive "can only concatenate str 
(not bytes) to str." would be clearer and educate users better.

As Serhiy said, the exact equivalent cannot be said for bytes.

>>> b'a' + bytearray(b'b')
b'ab'
>>> b'a' + memoryview(b'b')
b'ab'

However "Can only concatenate bytes, bytearray, or memoryview (not x) to 
bytes." would be good if this is in fact complete. I need to be educated on 
this ;-)  I was not sure about memoryview until I tried it.

--
nosy: +terry.reedy
type: behavior -> enhancement
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests of test_logging when bind() raises PermissionError (non-root user on Android)


Vinay Sajip added the comment:

I've added a patch that should handle these errors (including the 
SysLogHandlerTest, which wasn't reported, but I think the same logic applies).

To simulate failure during setup, uncomment one or more of the lines which says

# raise ValueError('dummy error raised')

--
assignee:  -> vinay.sajip
keywords: +patch
stage: needs patch -> patch review
Added file: http://bugs.python.org/file46180/issue-29177-01.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29181] skip tests that raise PermissionError in test_tarfile (non-root user on Android)


Changes by Xavier de Gaye :


--
nosy: +lars.gustaebel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29030] argparse: choices override metavar

2017-01-06 Thread paul j3


paul j3 added the comment:

Here's the method in HelpFormatter that creates the metavar:

def _metavar_formatter(self, action, default_metavar):
if action.metavar is not None:
result = action.metavar
elif action.choices is not None:
choice_strs = [str(choice) for choice in action.choices]
result = '{%s}' % ','.join(choice_strs)
else:
result = default_metavar

def format(tuple_size):
if isinstance(result, tuple):
return result
else:
return (result, ) * tuple_size
return format

So in order of priority it uses: 

the explicit metavar parameter
formatted choices
a default metavar (normally derived from the dest)

The MetavarTypeHelpFormatter subclass changes how that default is derived.  In 
the same spirit, you could write your own Formatter subclass that changes the 
above method, and its priority order.

In your example, the use of choices in the 'usage' like looks good to me.  I 
agree that its use in the 'help' line does not look so good.  But don't forget 
that for your end user, the positional 'dest' (or name) has no value.  Only you 
as the programmer sees and uses it.

This is one of many implementation details that could be included in the 
argparse docs.  But for some users those docs are already too complex.  The 
existing docs are not a formal module reference.

--
nosy: +paul.j3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28401] Don't support the PEP384 stable ABI in pydebug builds

2017-01-06 Thread Dmitry Shachnev


Dmitry Shachnev added the comment:

[Matthias Klose (doko) 2016-10-27 15:45]
> I'm not sure that you really want this, because it would make it impossible 
> to build an extension for the stable ABI for a debug build.

It looks like that is already impossible:

/usr/include/python3.5dm/object.h:65:2: error: #error Py_LIMITED_API is 
incompatible with Py_DEBUG, Py_TRACE_REFS, and Py_REF_DEBUG
 #error Py_LIMITED_API is incompatible with Py_DEBUG, Py_TRACE_REFS, and 
Py_REF_DEBUG
  ^

So in my opinion Stefano's patch makes sense.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29006] 2.7.13 _sqlite more prone to "database table is locked"

2017-01-06 Thread Larry Hastings


Larry Hastings added the comment:

FYI I'm keeping an eye on this for possible cherry-picking into 3.5.3 final, 
depending on the resolution.  Reverting 030e100f048a work for me, assuming 
that's a reasonable solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue

2017-01-06 Thread Ezio Melotti


Ezio Melotti added the comment:

test

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26865] Meta-issue: support of the android platform


Xavier de Gaye added the comment:

issue #29176: /tmp does not exist on Android and is used by 
curses.window.putwin()
issue #29177: skip tests of test_logging when bind() raises PermissionError 
(non-root user on Android)
issue #29180: skip tests that raise PermissionError in test_os (non-root user 
on Android)
issue #29181: skip tests that raise PermissionError in test_tarfile (non-root 
user on Android)
issue #29184: skip tests of test_socketserver when bind() raises 
PermissionError (non-root user on Android)
issue #29185: test_distutils fails on Android API level 24

--
dependencies: +/tmp does not exist on Android and is used by 
curses.window.putwin(), skip tests of test_logging when bind() raises 
PermissionError (non-root user on Android), skip tests of test_socketserver 
when bind() raises PermissionError (non-root user on Android), skip tests that 
raise PermissionError in test_os (non-root user on Android), skip tests that 
raise PermissionError in test_tarfile (non-root user on Android), 
test_distutils fails on Android API level 24

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29125] Shell injection via TIX_LIBRARY when using tkinter.tix

2017-01-06 Thread Larry Hastings


Larry Hastings added the comment:

I don't understand the fix.  Does this really prevent the injection?

I would fix it this way:

if tixlib is not None and os.path.exists(tixlib):

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29185] test_distutils fails on Android API level 24


New submission from Xavier de Gaye:

==
ERROR: test_tarfile_vs_tar 
(distutils.tests.test_archive_util.ArchiveUtilTestCase)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/tests/test_archive_util.py",
 line 170, i
n test_tarfile_vs_tar
spawn(gzip_cmd)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/spawn.py", line 36, 
in spawn
_spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/spawn.py", line 
159, in _spawn_posix
% (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gzip' failed with exit status 1

==
FAIL: test_copy_file_hard_link (distutils.tests.test_file_util.FileUtilTestCase)
--
Traceback (most recent call last):
  File 
"/sdcard/org.bitbucket.pyona/lib/python3.7/distutils/tests/test_file_util.py", 
line 88, in te
st_copy_file_hard_link
self.assertTrue(os.path.samestat(st2, st3), (st2, st3))
AssertionError: False is not true : (os.stat_result(st_mode=33206, 
st_ino=15948, st_dev=64800, st_nl
ink=1, st_uid=2000, st_gid=2000, st_size=12, st_atime=1483691935, 
st_mtime=1483691935, st_ctime=1483
691935), os.stat_result(st_mode=33206, st_ino=15949, st_dev=64800, st_nlink=1, 
st_uid=2000, st_gid=2
000, st_size=12, st_atime=1483691935, st_mtime=1483691935, st_ctime=1483691935))

--
Ran 236 tests in 1.885s

FAILED (failures=1, errors=1, skipped=38)
test test_distutils failed

--
assignee: xdegaye
components: Tests
messages: 284850
nosy: dstufft, eric.araujo, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: test_distutils fails on Android API level 24
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19977] Use "surrogateescape" error handler for sys.stdin and sys.stdout on UNIX for the C locale


Sworddragon added the comment:

Bug #28180 has caused me to make a look at the "encoding" issue this and the 
tickets before have tried to solve more or less. Being a bit unsure what the 
root cause and intention for all this was I'm now at a point to actually check 
this ticket. Here is an example code (executed with Python 3.5.3 RC1 by having 
LANG set to C):

import sys
sys.stdout.write('ä')


I thought with the surrogateescape error handler now being used for sys.stdout 
this would not throw an exception but I'm getting this:

UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 0: 
ordinal not in range(128)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29175] Tutorial links to file object methods are broken.

2017-01-06 Thread Jonathan Roach


Jonathan Roach added the comment:

OK, I understand that the older versions aren't going to be revised - that 
makes sense.

I think part of the reason I submitted this is the reader's path from the 
open() documentation to the most relevant part of the io documentation (the 
interface) is long and narrow: open()->file object(in the Glossary)->io(see 
note)..and then scroll several pages down.
Note: the link to io from the 'file object' glossary entry is visually tiny, 
and in last paragraph - effectively making it almost invisible. This also seems 
to be the only place in the chain of links from open() which gets you to the io 
documentation.
When trying to find the interface documentation I tried the visually bigger, 
earlier links and ended up like I was being given the run-around through the 
various glossary entries.
As a documentation user I'd ideally like a direct link (which is a bit visually 
bigger than 'io') from open() to the relevant, interface section of the io 
documentation. However, upstanding you have a better feel for the tone of the 
documentation, I can see you might take a different route.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests of test_logging when bind() raises PermissionError (non-root user on Android)


Xavier de Gaye added the comment:

> Do you want to take this on, or do you want me to look at it?

I would be very grateful if you would handle that :)
But if you cannot spare the time, I can give it a try.
One point I forgot to mention is that DatagramHandlerTest and SysLogHandlerTest 
also fail with PermissionError when run as their subclass 
UnixDatagramHandlerTest (resp. UnixSysLogHandlerTest).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests of test_logging when bind() raises PermissionError (non-root user on Android)


Xavier de Gaye added the comment:

I have split this issue, issue 29184 is for fixing the tests on 
test_socketserver and this issue is for fixing the tests on test_logging.

--
assignee: xdegaye -> 
title: skip tests using socketserver.UnixStreamServer when bind() raises 
PermissionError -> skip tests of test_logging when bind() raises 
PermissionError (non-root user on Android)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29184] skip tests of test_socketserver when bind() raises PermissionError (non-root user on Android)


New submission from Xavier de Gaye:

This happens on Android for a non-root user.  Multiple tests fail in 
test_socketserver with identical backtraces, only the first one is listed here.

==  
  [905/2616]
ERROR: test_ForkingUnixDatagramServer (test.test_socketserver.SocketServerTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", 
line 243, in test_Fork
ingUnixDatagramServer
self.dgram_examine)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/support/__init__.py", 
line 2040, in decorator
return func(*args)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", 
line 121, in run_serve
r
svrcls, hdlrbase)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_socketserver.py", 
line 114, in make_serv
er
server = MyServer(addr, MyHandler)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 452, 
in __init__
self.server_bind()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/socketserver.py", line 466, 
in server_bind
self.socket.bind(self.server_address)
PermissionError: [Errno 13] Permission denied

--
assignee: xdegaye
components: Tests
messages: 284845
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests of test_socketserver when bind() raises PermissionError 
(non-root user on Android)
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29183] Unintuitive error handling in wsgiref when a crash happens in write() or close()

2017-01-06 Thread Jerome Leclanche

New submission from Jerome Leclanche:

TLDR: When an error happens in the wsgiref's write() or close(), stack traces
get inundated with irrelevant yet legitimate errors which make it hard to track
down the real issue.

Couple of examples of this happening in practice:
https://stackoverflow.com/questions/28124461/how-do-i-solve-nonetype-object-is-not-callable-with-beautifulsoup-and-bottle

https://stackoverflow.com/questions/27518844/error-when-serving-html-with-wsgi

How to reproduce: The file I've attached reproduces the error on python 3.4,
3.5 and 3.6. The handler returns a string instead of bytes, which fails an
early assert in handlers.py: write(data).

BaseHandler.run() triggers, gets as far as finish_response(), which triggers
the above AssertionError. It falls into the except: block which attempts to
handle_error(). But before doing that, it triggers self.close(), which sets
result/headers/status/environ to None, bytes_sent to 0 and headers_sent to
False.

Now when handle_error() triggers, `self.headers_sent` is False because of that,
which attempts to trigger finish_response() *again*. This triggers a write()
which attempts sending the headers, which checks client_is_modern(),
subscripting `self.environ` which at that point has already been set to None.
New error, which is caught in run()'s except block and re-raised after closing
the connection (again).

I probably skipped some steps because the traceback is truly a mess. I think
this could be improved, if only so that it doesn't get so confusing anymore.

--
files: test_server.py
messages: 284844
nosy: jleclanche
priority: normal
severity: normal
status: open
title: Unintuitive error handling in wsgiref when a crash happens in write() or
close()
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file46179/test_server.py

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29116] Make str and bytes error messages on concatenation conform with other sequences


Changes by Raymond Hettinger :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2771] Test issue

2017-01-06 Thread Brett Cannon


Changes by Brett Cannon :


--
pull_requests: +9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista

2017-01-06 Thread Segev Finer


Segev Finer added the comment:

Second version of the patch after review by eryksun.

Please pay attention to the hack in _execute_child due to having to 
temporarily override the handle_list if the user supplied
one.

As for pass_fds: as you noted, it has it's own share of complexities and issues 
and I think it's best to leave it to a separate patch/issue.

--
Added file: 
http://bugs.python.org/file46178/windows-subprocess-close-fds-v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29070] Integration tests for pty.spawn on Linux and all other platforms

2017-01-06 Thread Cornelius Diekmann


Cornelius Diekmann added the comment:

Thank you Martin very much for this very helpful review. I updated and 
simplified the tests and implemented your suggestions.

There are three open issues left.

1)
> It looks like you depend on fixing Issue 26228, but the patch there will 
> conflict with your changes. Maybe merge with the other patch, or propose an 
> alternative fix.

I reverted my changes to pty.py, so the patches no longer conflict for this 
file. Currently, my test suite reveals the pty bug on FreeBSD but does not fix 
it. Since I'm not a FreeBSD expert and I don't want to steal Chris' patch 
[issue26228], I'd like to keep those issues separate. In other words, I'm 
proposing only a testsuite which introduces no behavioral changes and uncovers 
a bug.

This also means that this test suite is currently hanging on FreeBSD and OS X. 
The good news is, the patch in issue26228 fixes the problem.

> The documentation currently mentions the code is only tested on Linux, so it 
> would be nice to update that.
Is it okay to keep this hanging around until the pty module is no longer broken 
on FreeBSD?

2)
> Why does the patch slow the tests down so much? Ideally, it is nice to keep 
> the tests as fast as possible.

The test suite does some heavy integration testing. The test in the class 
PtyWhiteBoxIntegrationReadSlaveTest are the slow ones. The test suite reads a 
large amount of data from the spawned child process. The amount of data was 
chosen to be a few kB in size to make sure that we won't lose data of the child 
and that --in case of a bug in the code-- we don't accidentally succeed due to 
race conditions.

Is 1-2 seconds really too slow? We can reduce the amount of data, but increase 
the risk that a future code change introduces a race condition which is not 
covered by the test suite.


3)
About the system bell pretty printing test:
> This seems platform-dependent. Do you really need to test echoing of control
> characters? We only need to test Python, not the operating system.

Including these integration tests has advantages and disadvantages.

Advantages:
 * Integration tests which deliberately depend on the careful interplay of many 
modules make sure that the overall system is healthy.
 * Modules such as termios or terminal-specific parts of os seem not to be 
tested currently. This test suite depends on their correct functionality and is 
thus a high-level test for them. In addition, pty.spawn() is the perfect place 
to test them.
 * The tests test actual terminal features. Without, we could not uncover if 
e.g., a pty master/slave is replaced by a simple pipe.
 * Tests are set up by increasing order of complexity w.r.t. the underlying 
modules they depend on. If something breaks in the future, these tests are 
designed to ease debugging and directly point to the error.
 * They provide some documentation about the magic of terminals and examples.
 * Those tests are fast.

Disadvantages:
 * Control-Character pretty printing only works on Linux.
 * Relies on the (POSIX-compliant) internals of the operating system.

I wanted to include some tests which depend on some terminal-specific features 
to make sure that the complete setup works and is operational. For example, the 
test suite now documents the difference between ptys and pipes and would 
complain if any code change breaks terminal-specific features. I chose control 
character pretty printing in some tests because this is one of the simplest 
features of terminals to test, debug, and verify. In contrast, for example, it 
would be extremely fragile and probably complete overkill to test things such 
as terminal delays. 

My personal feeling says that the advantages outweigh the disadvantages. If you 
disagree, we can remove them :-)

--
Added file: http://bugs.python.org/file46177/pty_tests.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError


Vinay Sajip added the comment:

> Would that make sense to move the server.start() part out of setUp() and in 
> its own method

My preference would be to just catch the error in SocketHandlerTest.setUp() and 
leave things in a tidy state (e.g. .server and .sock_hdlr set to None), make 
the tearDown() logic take that into account, and in each test just skip if 
.server is None.

Do you want to take this on, or do you want me to look at it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29116] Make str and bytes error messages on concatenation conform with other sequences

2017-01-06 Thread Ivan Levkivskyi


Changes by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28961] unittest.mock._Call ignores `name` parameter


STINNER Victor added the comment:

I applied the latest mock.patch to Python 3.6 and default (future 3.7). I 
prefer to wait for the 3.5.3 release before backporting the fix to 3.5, the fix 
is minor, I don't want to annoy the release manager yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28961] unittest.mock._Call ignores `name` parameter


Roundup Robot added the comment:

New changeset 50424a903593 by Victor Stinner in branch '3.6':
Fix unittest.mock._Call: don't ignore name
https://hg.python.org/cpython/rev/50424a903593

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29178] Adding bytes.frombuffer(byteslike) constructor

2017-01-06 Thread Yury Selivanov


Yury Selivanov added the comment:

> This is just a two-liner:
>
>with memoryview(bytelike) as m:
>bs = bytes(m[start:end])

Which virtually no one follows :(


> Adding new method to builtin type has high bar. I doubts that there are 
> enough use cases in which bytes.frombuffer() has an advantage.

Any protocol parsing code has a lot of slicing.

> The signature of bytes.frombuffer() looks questionable. Why length and offset 
> instead of start and stop indices as in slices? Why length is first and 
> offset is last? This contradicts the interface of Python 2 buffer(), 
> socket.sendfile(), os.sendfile(), etc.

I propose to make both arguments keyword-only.

> There is also a problem with returned type for subclasses (this is always a 
> problem for alternate constructors).

Good point. How do we usually solve this in CPython?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29178] Adding bytes.frombuffer(byteslike) constructor


Serhiy Storchaka added the comment:

Count me as -1 too.

This is just a two-liner:

with memoryview(bytelike) as m:
bs = bytes(m[start:end])

In most cases, when all content is used, the bytes constructor works fine.

bs = bytes(bytelike)

This works not just with bytes, but with bytearray and most other bytes-like 
arrays. With frombuffer() you need to add a new class method to all these 
classes.

Adding new method to builtin type has high bar. I doubts that there are enough 
use cases in which bytes.frombuffer() has an advantage.

The signature of bytes.frombuffer() looks questionable. Why length and offset 
instead of start and stop indices as in slices? Why length is first and offset 
is last? This contradicts the interface of Python 2 buffer(), 
socket.sendfile(), os.sendfile(), etc.

There is also a problem with returned type for subclasses (this is always a 
problem for alternate constructors). Should B.frombuffer() where B is a bytes 
subclass return an instance of bytes or B? If it always returns a bytes object, 
we need to use a constructor for subclasses, if it returns an instance of a 
subclass, how can it be implemented efficiently?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29182] Remove the warning in urllib docs that it doesn't do certificate validate by default.

2017-01-06 Thread Senthil Kumaran


New submission from Senthil Kumaran:

It started as a discussion in this issue: 
http://bugs.python.org/issue22417#msg284604

I think, that warning can be removed.  If no one has any objections, I will 
commit this attached patch.

--
assignee: orsenthil
components: Documentation
files: remove_warning.patch
keywords: patch
messages: 284836
nosy: benjamin.peterson, christian.heimes, clopez, orsenthil
priority: normal
severity: normal
stage: patch review
status: open
title: Remove the warning in urllib docs that it doesn't do certificate 
validate by default.
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file46176/remove_warning.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29158] Possible glitch in the interaction of a thread and a multiprocessing manager

2017-01-06 Thread luke_16


luke_16 added the comment:

Regarding Davin's last paragraph: "Without pulling apart your code...", I would 
like to point out that what I'm doing is what the Documentation instructs:

https://docs.python.org/2/library/multiprocessing.html#using-a-remote-manager

So, I want to access a process running in another machine, which is the 
server.py, from another machine running client.py. But if I want tho run both 
separately in the same machine, I should also be able to.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista

2017-01-06 Thread Eryk Sun


Eryk Sun added the comment:

Implementing pass_fds on Windows is a problem if Popen has to implement the 
undocumented use of the STARTUPINFO cbReserved2 and lpReserved2 fields to 
inherit CRT file descriptors. I suppose we could implement this ourselves in 
_winapi since it's unlikely that the data format will ever change. Just copy 
what the CRT's accumulate_inheritable_handles() function does, but constrained 
by an array of file descriptors.

--
components: +Library (Lib)
nosy: +eryksun
stage:  -> patch review
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29178] Adding bytes.frombuffer(byteslike) constructor

2017-01-06 Thread Yury Selivanov


Yury Selivanov added the comment:

I've added a couple of review comments.  Also, it looks like you can count 
Antoine Pitrou as +1 too.

Two questions:

1. length or count?  Need to look through builtins/stdlib and see what is more 
common in CPython.

2. Maybe we should make length/count and offset keyword-only arguments?

--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29181] skip tests that raise PermissionError in test_tarfile (non-root user on Android)


New submission from Xavier de Gaye:

==  
  [339/2616]
ERROR: test_link_size (test.test_tarfile.Bz2WriteTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 
1102, in test_link_siz
e
os.link(target, link)
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/link_target' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/link'

==
ERROR: test_link_size (test.test_tarfile.GzipWriteTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 
1102, in test_link_siz
e
os.link(target, link)
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/link_target' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/link'

==
ERROR: test_add_hardlink (test.test_tarfile.HardlinkTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 
1563, in setUp
os.link(self.foo, self.bar)
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/foo' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/bar'

==
ERROR: test_add_twice (test.test_tarfile.HardlinkTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 
1563, in setUp
os.link(self.foo, self.bar)
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/foo' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/bar'

==
ERROR: test_dereference_hardlink (test.test_tarfile.HardlinkTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 
1563, in setUp
os.link(self.foo, self.bar)
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/foo' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/bar'

==
ERROR: test_link_size (test.test_tarfile.WriteTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_tarfile.py", line 
1102, in test_link_siz
e
os.link(target, link)
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp-tard
ir/link_target' -> '/data/local/tmp/test_python_2295/@test_2295_tmp-tardir/link'

--
Ran 426 tests in 17.296s

FAILED (errors=6, skipped=80)
test test_tarfile failed

--
assignee: xdegaye
components: Tests
messages: 284832
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests that raise PermissionError in test_tarfile (non-root user on 
Android)
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError


Xavier de Gaye added the comment:

> My understanding is that skipTest would normally be raised in the test method 
> itself or as a decorator to it, and not in setUp() itself.

Agreed. Would that make sense to move the server.start() part out of setUp() 
and in its own method that would be called by each test (I am not very familiar 
with test_logging) ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29168] multiprocessing pickle error


Vinay Sajip added the comment:

I'm closing this on the basis that I don't believe there's a bug here, but you 
can of course re-open if you have evidence to the contrary.

--
resolution:  -> not a bug
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22135] allow to break into pdb with Ctrl-C for all the commands that resume execution


Xavier de Gaye added the comment:

The code review of your first patch still applies to your last patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError


Vinay Sajip added the comment:

> To reproduce the test_logging cleanup problem, insert skipTest() in setUp()

My understanding is that skipTest would normally be raised in the test method 
itself or as a decorator to it, and not in setUp() itself. (It wouldn't make 
sense to, as that would skip every test in the test case - not the obvious 
thing to do.) I can understand that failures that happen in setUp() may cause 
tearDown() not to be called.

I would guess that setUp() should recover from any problem and set a flag which 
is then used to skip in the other tests (which rely on that problem not being 
there).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError


Xavier de Gaye added the comment:

The test_logging cleanup problem induces another problem. When test_lib2to3 is 
run after the failing test_logging then test_lib2to3 fails with:

==
FAIL: test_filename_changing_on_output_single_dir 
(lib2to3.tests.test_main.TestMain)
2to3 a single directory with a new output dir and suffix.
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/lib2to3/tests/test_main.py", 
line 91, in test_file
name_changing_on_output_single_dir
self.py3_dest_dir, self.py2_src_dir), stderr)
AssertionError: "Output in '/data/local/tmp/tmp4eyn96tg/python3_project' will 
mirror the input direc
tory '/data/local/tmp/tmp4eyn96tg/python2_project' layout" not found in 
'WARNING: --write-unchanged-
files/-W implies -w.\n'

--
Ran 616 tests in 140.986s

FAILED (failures=1, expected failures=1)
Warning -- logging._handlerList was modified by test_lib2to3
test test_lib2to3 failed


It seems that this is related to the test_logging failure because the expected 
string "Output in '/data/local/tmp/tmp4eyn96tg/python3_project' will mirror the 
input directory '/data/local/tmp/tmp4eyn96tg/python2_project' layout" is output 
by logger.info() in Lib/lib2to3/main.py:243.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError


Xavier de Gaye added the comment:

To reproduce the test_logging cleanup problem, insert skipTest() in setUp():

diff -r 4a97fa319bf7 Lib/test/test_logging.py
--- a/Lib/test/test_logging.py  Fri Jan 06 09:52:19 2017 +0100
+++ b/Lib/test/test_logging.py  Fri Jan 06 16:39:38 2017 +0100
@@ -1440,6 +1440,7 @@
 """Set up a TCP server to receive log messages, and a SocketHandler
 pointing to that server's address and port."""
 BaseTest.setUp(self)
+self.skipTest('Skip test in setUp().')
 self.server = server = self.server_class(self.address,
  self.handle_socket, 0.01)
 server.start()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29175] Tutorial links to file object methods are broken.

2017-01-06 Thread R. David Murray


R. David Murray added the comment:

The tutorial links are definitely bugs, and the section may need some tweaking. 
 The 'file object' documentation is replaced by the io module, which completely 
defines the file object API.  There is a link from the 'file object' glossary 
entry to the IO module indicating this.

Only 3.6 and 3.7 docs get documentation updates at this point, so I've removed 
the others from versions (we use versions to say where things will get fixed).  
(Well, 3.5 docs can still get updates at the moment, but that won't last long, 
final is soon.)

--
nosy: +r.david.murray
title: Documentation for File objects missing -> Tutorial links to file object 
methods are broken.
versions:  -Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29180] skip tests that raise PermissionError in test_os (non-root user on Android)


New submission from Xavier de Gaye:

==  
 [1633/2616]
ERROR: test_link (test.test_os.LinkTests)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1703, 
in test_link
self._test_link(self.file1, self.file2)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1698, 
in _test_link
os.link(file1, file2)
PermissionError: [Errno 13] Permission denied: '@test_2295_tmp' -> 
'@test_2295_tmp2'

==
ERROR: test_link_bytes (test.test_os.LinkTests)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1707, 
in test_link_bytes
bytes(self.file2, sys.getfilesystemencoding()))
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1698, 
in _test_link
os.link(file1, file2)
PermissionError: [Errno 13] Permission denied: b'@test_2295_tmp' -> 
b'@test_2295_tmp2'

==
ERROR: test_unicode_name (test.test_os.LinkTests)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1717, 
in test_unicode_name
self._test_link(self.file1, self.file2)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 1698, 
in _test_link
os.link(file1, file2)
PermissionError: [Errno 13] Permission denied: '@test_2295_tmpñ' -> 
'@test_2295_tmpñ2'

==
ERROR: test_stty_match (test.test_os.TermsizeTests)
Check if stty returns the same results
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 2795, 
in test_stty_match
size = subprocess.check_output(['stty', 'size']).decode().split()
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 336, in 
check_output
**kwargs).stdout
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 403, in 
run
with Popen(*popenargs, **kwargs) as process:
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 707, in 
__init__
restore_signals, start_new_session)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/subprocess.py", line 1323, in 
_execute_child
raise child_exception_type(errno_num, err_msg)
PermissionError: [Errno 13] Permission denied

==
ERROR: test_attributes (test.test_os.TestScandir)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_os.py", line 3150, 
in test_attributes
os.link(filename, os.path.join(self.path, "link_file.txt"))
PermissionError: [Errno 13] Permission denied: 
'/data/local/tmp/test_python_2295/@test_2295_tmp/file
.txt' -> '/data/local/tmp/test_python_2295/@test_2295_tmp/link_file.txt'

--
Ran 238 tests in 4.675s

FAILED (errors=5, skipped=40)
test test_os failed

--
assignee: xdegaye
components: Tests
messages: 284824
nosy: xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: skip tests that raise PermissionError in test_os (non-root user on 
Android)
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28961] unittest.mock._Call ignores `name` parameter

2017-01-06 Thread Michael Foord


Michael Foord added the comment:

Yep, LGTM as well. Nicely spotted!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27876] Add SSLContext.set_version_range(minver, maxver=None)

2017-01-06 Thread Christian Heimes


Christian Heimes added the comment:

PoC implementation:

from enum import Enum
import ssl

OP_NO_TLSv1_3 = getattr(ssl, 'OP_NO_TLSv1_3', 0)

OP_NO_FLAGS = [
ssl.OP_NO_SSLv2,
ssl.OP_NO_SSLv3,
ssl.OP_NO_TLSv1,
ssl.OP_NO_TLSv1_1,
ssl.OP_NO_TLSv1_2,
OP_NO_TLSv1_3
]

OP_NO_MASK = sum(OP_NO_FLAGS)


class TLSVersions(Enum):
SSLv2 = 'SSL 2.0', 0x0200, 0
SSLv3 = 'SSL 3.0', 0x0300, 1
TLSv1 = 'TLS 1.0', 0x0301, 2
TLSv1_1 = 'TLS 1.1', 0x0302, 3
TLSv1_2 = 'TLS 1.2', 0x0303, 4

if OP_NO_TLSv1_3:
TLSv1_3 = 'TLS 1.3', 0x0304, 5
MAX = TLSv1_3
else:
MAX = TLSv1_2

MIN = TLSv1

def __init__(self, prettyname, wireprotocol, offset):
self.prettyname = prettyname
self.wireprotocol = wireprotocol
self.noflag = OP_NO_FLAGS[offset]
self.minflag = sum(OP_NO_FLAGS[:offset])
self.maxflag = sum(OP_NO_FLAGS[offset+1:])

def __repr__(self):
return ("<{0.__class__.__name__}.{0.name} "
"({0.prettyname}, 0x{0.wireprotocol:x})>").format(self)

__str__ = __repr__


class SSLContext(ssl.SSLContext):
def set_version(self, minver=TLSVersions.MIN, maxver=TLSVersions.MAX):
options = self.options & ~OP_NO_MASK
self.options = options | minver.minflag | maxver.maxflag


if __name__ == '__main__':
for name, member in TLSVersions.__members__.items():
print(name, member)

ctx = SSLContext(ssl.PROTOCOL_SSLv23)
print(ctx.options)
ctx.set_version(minver=TLSVersions.SSLv3, maxver=TLSVersions.TLSv1_1)
print(ctx.options)

--
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29179] Py_UNUSED is not documented

2017-01-06 Thread Petr Viktorin


New submission from Petr Viktorin:

The Py_UNUSED macro, which was added in Python 3.4, is not documented.
Is this an omission, or is it undocumented on purpose?
I can prepare a patch if it's the former.


The macro was added in: http://bugs.python.org/issue19976
and referenced in: http://bugs.python.org/issue26179
Usage on Github: https://github.com/search?q=py_unused=Code

--
assignee: docs@python
components: Documentation
messages: 284821
nosy: docs@python, encukou
priority: normal
severity: normal
status: open
title: Py_UNUSED is not documented
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22135] allow to break into pdb with Ctrl-C for all the commands that resume execution

2017-01-06 Thread Chun-Yu Tseng


Chun-Yu Tseng added the comment:

Ping :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29176] /tmp does not exist on Android and is used by curses.window.putwin()

2017-01-06 Thread Chi Hsuan Yen


Chi Hsuan Yen added the comment:

I guess replace mkstemp (C function) with tempfile.mkstemp (Python function) 
can solve the problem.

--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22231] httplib: unicode url will cause an ascii codec error when combined with a utf-8 string header

2017-01-06 Thread Herman Schistad


Herman Schistad added the comment:

I can confirm that this patch solves the issues I've had where I can submit 
multipart forms provided I have a string URL, but not if it's unicode.

I'm using Python 2.7.12. Applying the patch fixes the issue.

Code which breaks, assuming the file contains binary data:


# -*- encoding: utf-8 -*-
import urllib3
pool_manager = urllib3.PoolManager(num_pools=2)
url = u'http://example.org/form' # removing the 'u' fixes it
content = open('/some/binary/file').read()
fields = [
('foo', 'something'),
('bar', ('/some/binary/file', content, 'application/octet-stream'))
]
pool_manager.request("POST", url, fields=fields, encode_multipart=True, 
headers={})

--
nosy: +Herman Schistad

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19764] subprocess: use PROC_THREAD_ATTRIBUTE_HANDLE_LIST with STARTUPINFOEX on Windows Vista

2017-01-06 Thread Segev Finer


Segev Finer added the comment:

Though Python has taken measures to mark handles as non-inheritable there is 
still a possible race due to having to create inheritable handles while 
creating processes with stdio pipes (subprocess).

Attached is a Patch that implements subprocess.Popen(close_fds=True) with stdio 
handles on Windows using PROC_THREAD_ATTRIBUTE_HANDLE_LIST, which plugs that 
race completely.

I implemented this by adding the attribute STARTUPINFO._handleList, which when 
passed to _winapi.CreateProcess, will be passed to CreateProcess as a 
PROC_THREAD_ATTRIBUTE_HANDLE_LIST. subprocess.py can than use this attribute as 
needed with inherit_handles=True to only inherit the stdio handles.

The STARTUPINFO._handleList attribute can also be used to implement pass_fds 
later on. Though the exact behavior of how to convert a file descriptor list to 
a handle list might be a bit sensitive, so I left that out for now.

This patch obviously doesn't support Windows XP but Python 3 doesn't support XP 
anymore either.

--
keywords: +patch
nosy: +Segev Finer
Added file: http://bugs.python.org/file46175/windows-subprocess-close-fds.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29054] pty.py: pty.spawn hangs after client disconnect over nc (netcat)

2017-01-06 Thread Cornelius Diekmann


Cornelius Diekmann added the comment:

[no status change, this issue currently does NOT need any attention]

To keep issues separate, I just wanted to document a comment about this issue 
mentioned in issue29070. It refers to the _copy loop.

if STDIN_FILENO in rfds:
 data = stdin_read(STDIN_FILENO)
 if not data:
 fds.remove(STDIN_FILENO)
+# Proposal for future behavior change: Signal EOF to
+# slave if STDIN of master is gone. Solves issue29054.
+# os.write(master_fd, b'\x04')
 else:
 _writen(master_fd, data)

> vadmium 2017/01/04 21:50:26
> I suggest leaving this for the other [issue29054, i.e. this] bug. Another 
> option may be to send SIGHUP
> (though I am far from an expert on Unix terminals :).

http://bugs.python.org/review/29070/diff/19626/Lib/pty.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29132] shlex.shlex with punctuation_chars and posix doesn't handle punctuation next to quotes

2017-01-06 Thread Evan


Evan added the comment:

I've just submitted the form.

I'm attaching a second patch which also addresses another similar bug 
demonstrated in the test case. The fix is to check for the 'c' state before 
checking for quotes or escape characters.

--
Added file: http://bugs.python.org/file46174/shlex-posix-quote2.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError


Xavier de Gaye added the comment:

test_logging fails also with the following backtrace:

==
FAIL: test_output (test.test_logging.UnixSocketHandlerTest)
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 
1527, in setUp
SocketHandlerTest.setUp(self)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 
1442, in setUp
BaseTest.setUp(self)
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_logging.py", line 
111, in setUp
raise AssertionError('Unexpected handlers: %s' % hlist)
AssertionError: Unexpected handlers: []

This is because when a test fails or when it is skipped, the 
BaseTest.tearDown() method is not called and as a consequence the following 
test fails.

--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29176] /tmp does not exist on Android and is used by curses.window.putwin()


New submission from Xavier de Gaye:

==
ERROR: test_module_funcs (test.test_curses.TestCurses)
Test module-level functions
--
Traceback (most recent call last):
  File "/sdcard/org.bitbucket.pyona/lib/python3.7/test/test_curses.py", line 
219, in test_module_fun
cs
self.stdscr.putwin(f)
FileNotFoundError: [Errno 2] No such file or directory: 
'/tmp/py.curses.putwin.nsIZYY'

--

--
assignee: xdegaye
components: Extension Modules
keywords: patch
messages: 284810
nosy: twouters, xdegaye
priority: normal
severity: normal
stage: needs patch
status: open
title: /tmp does not exist on Android and is used by curses.window.putwin()
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29177] skip tests using socketserver.UnixStreamServer when bind() raises PermissionError