[issue29842] Make Executor.map work with infinite/large inputs correctly

[Max]

Max added the comment:

I'm also concerned about this (undocumented) inconsistency between map and 
Executor.map.

I think you would want to make your PR limited to `ThreadPoolExecutor`. The 
`ProcessPoolExecutor` already does everything you want with its `chunksize` 
paramater, and adding `prefetch` to it will jeopardize the optimization for 
which `chunksize` is intended.

Actually, I was even thinking whether it might be worth merging `chunksize` and 
`prefetch` arguments. The semantics of the two arguments is similar but not 
identical. Specifically, for `ProcessPoolExecutor`, there is pretty clear 
pressure to increase the value of `chunksize` to reduce amortized IPC costs; 
there is no IPC with threads, so the pressure to increase `prefetch` is much 
more situational (e.g., in the busy pool example I give below).

For `ThreadPoolExecutor`, I prefer your implementation over the current one, 
but I want to point out that it is not strictly better, in the sense that *with 
default arguments*, there are situations where the current implementation 
behaves better.

In many cases your implementation behaves much better. If the input is too 
large, it prevents out of memory condition. In addition, if the pool is not 
busy when `map` is called, your implementation will also be faster, since it 
will submit the first input for processing earlier.

But consider the case where input is produced slower than it can be processed 
(`iterables` may fetch data from a database, but the callable `fn` may be a 
fast in-memory transformation). Now suppose the `Executor.map` is called when 
the pool is busy, so there'll be a delay before processing begins. In this 
case, the most efficient approach is to get as much input as possible while the 
pool is busy, since eventually (when the pool is freed up) it will become the 
bottleneck. This is exactly what the current implementation does.

The implementation you propose will (by default) only prefetch a small number 
of input items. Then when the pool becomes available, it will quickly run out 
of prefetched input, and so it will be less efficient than the current 
implementation. This is especially unfortunate since the entire time the pool 
was busy, `Executor.map` is just blocking the main thread so it's literally 
doing nothing useful.

Of course, the client can tweak `prefetch` argument to achieve better 
performance. Still, I wanted to make sure this issue is considered before the 
new implementation is adopted.

>From the performance perspective, an even more efficient implementation would 
>be one that uses three background threads:

- one to prefetch items from the input
- one to sends items to the workers for processing
- one to yield results as they become available

It has a disadvantage of being slightly more complex, so I don't know if it 
really belongs in the standard library.

Its advantage is that it will waste less time: it fetches inputs without pause, 
it submits them for processing without pause, and it makes results available to 
the client as soon as they are processed. (I have implemented and tried this 
approach, but not in productioon.)

But even this implementation requires tuning. In the case with the busy pool 
that I described above, one would want to prefetch as much input as possible, 
but that may cause too much memory consumption and also possibly waste 
computation resources (if the most of input produced proves to be unneeded in 
the end).

--
nosy: +max

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Changes by Xiang Zhang :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30110] test_asyncio reports reference leak

[Xiang Zhang]

Changes by Xiang Zhang :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 982a17e02d99dcf6e4dff93110cff5ecc59247f5 by Xiang Zhang in branch 
'2.7':
bpo-30242: resolve some undefined behaviours in struct (#1418) (#1588)
https://github.com/python/cpython/commit/982a17e02d99dcf6e4dff93110cff5ecc59247f5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30110] test_asyncio reports reference leak

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 51ab4ada8fc99c24d434a533693375169c097006 by Xiang Zhang in branch 
'3.5':
bpo-30110: fix resource leak in test_asyncio.test_events (#1413) (#1585)
https://github.com/python/cpython/commit/51ab4ada8fc99c24d434a533693375169c097006


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset dd2a09cf98845b1460f0e049ad0ffeeb5c6c6476 by Xiang Zhang in branch 
'3.5':
bpo-30242: resolve some undefined behaviours in struct (#1418) (#1587)
https://github.com/python/cpython/commit/dd2a09cf98845b1460f0e049ad0ffeeb5c6c6476


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset aad1caf55f7fadb00ee097a1465bece4e128d8ef by Xiang Zhang in branch 
'3.6':
bpo-30242: resolve some undefined behaviours in struct (#1418) (#1586)
https://github.com/python/cpython/commit/aad1caf55f7fadb00ee097a1465bece4e128d8ef


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30110] test_asyncio reports reference leak

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 0ce1f7e02d4102e8a459d1855d689c8e4613a327 by Xiang Zhang in branch 
'3.6':
bpo-30110: fix resource leak in test_asyncio.test_events (#1413) (#1584)
https://github.com/python/cpython/commit/0ce1f7e02d4102e8a459d1855d689c8e4613a327


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29570] Windows Buildbot 2.7 is broken

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you Jeremy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1681

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1680

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1679

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30242] resolve undefined behaviour in struct

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 981096f98b9c131594b0ac85ad01b63cbd11aa0a by Xiang Zhang in branch 
'master':
bpo-30242: resolve some undefined behaviours in struct (#1418)
https://github.com/python/cpython/commit/981096f98b9c131594b0ac85ad01b63cbd11aa0a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30110] test_asyncio reports reference leak

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1678

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30110] test_asyncio reports reference leak

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1677

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30110] test_asyncio reports reference leak

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 7c278a5eeb656c2b48a85bbd761ce165f1751bb6 by Xiang Zhang in branch 
'master':
bpo-30110: fix resource leak in test_asyncio.test_events (#1413)
https://github.com/python/cpython/commit/7c278a5eeb656c2b48a85bbd761ce165f1751bb6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30224] remove outdated checks in struct

[Xiang Zhang]

Changes by Xiang Zhang :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30224] remove outdated checks in struct

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 96f502856796f9310fed7161dc540201a4afc1ee by Xiang Zhang in branch 
'master':
bpo-30224: remove outdated checks in struct (#1374)
https://github.com/python/cpython/commit/96f502856796f9310fed7161dc540201a4afc1ee


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30350] devguide suggests to use VS 2008 to build Python 2.7, but VS 2008 is no more supported?

[Zachary Ware]

Zachary Ware added the comment:

> I have a changeset that restores this ability, but would like to know if 
> that change was intentional, or just missed due to not having any buildbots 
> covering VS9.0-only builds.

It was intentional; it was more effort than I wanted to put in to get newer 
OpenSSL to work with both the new build scripts and the old method, so I 
figured installing Perl wasn't the worst thing in the world for those who 
wanted to use the VS9.0 files.  If your patch isn't too big or invasive, I'd be 
fine with applying it, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29570] Windows Buildbot 2.7 is broken

[Zachary Ware]

Zachary Ware added the comment:

Thank you Jeremy, we'll call this one closed now.  We should open a new issue 
about the VS9.0 builder failure, which as you say is entirely unrelated.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30354] Change data model documentation to zero-argument super()

[Nick Coghlan]

Nick Coghlan added the comment:


New changeset 12b1c180986fc744331b8f30d3d2f49a0fdb43dd by Nick Coghlan 
(csabella) in branch 'master':
bpo-30354: Update data model documentation for super() (GH-1561)
https://github.com/python/cpython/commit/12b1c180986fc744331b8f30d3d2f49a0fdb43dd


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30367] Cannot build CPython3.6 with module “testcapimodule” statically

[Nan Zhang]

New submission from Nan Zhang:

up vote
0
down vote
favorite
I am trying to build CPython3.6 from GitHub statically with the Documentation 
below:

https://wiki.python.org/moin/BuildStatically

Changed *shared* to *static*

And only uncommented the module "testcapimodule" in Modules/Setup file:

_testcapi _testcapimodule.c # Python C API test module

And ran with 'make LINKFORSHARED=" "'

However I got errors that:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 
-Wall -Wstrict-prototypes-std=c99 -Wextra -Wno-unused-result 
-Wno-unused-parameter -Wno-missing-field-initializers   -I. -I./Include
-DPy_BUILD_CORE  -c ./Modules/_testcapimodule.c -o Modules/_testcapimodule.o
./Modules/_testcapimodule.c: In function ‘test_datetime_capi’:
./Modules/_testcapimodule.c:2191:9: error: ‘PyDateTimeAPI’ undeclared 
(first use in this function)
 if (PyDateTimeAPI) {
 ^
./Modules/_testcapimodule.c:2191:9: note: each undeclared identifier is 
reported only once for each function it appears in
./Modules/_testcapimodule.c:2203:5: error: ‘PyDateTime_IMPORT’ 
undeclared (first use in this function)
 PyDateTime_IMPORT;
 ^
./Modules/_testcapimodule.c:2208:1: warning: control reaches end of 
non-void function [-Wreturn-type]
 }
 ^
make: *** [Modules/_testcapimodule.o] Error 1

But when I build Cpython3.6 with *shared*, it can succeed.

--
components: Build
messages: 293664
nosy: Nan Zhang
priority: normal
severity: normal
status: open
title: Cannot build CPython3.6 with module “testcapimodule” statically
type: compile error
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30350] devguide suggests to use VS 2008 to build Python 2.7, but VS 2008 is no more supported?

[Jeremy Kloth]

Jeremy Kloth added the comment:

Now that my buildbot is back online, there is one unresolved issue pertaining 
to using VS9.0 only for building.  That is building without Perl.  It was a 
supported feature prior to backporting the MSBuild toolchain.  With recent 
OpenSSL releases, without Perl is no longer functional.

I have a changeset that restores this ability, but would like to know if that 
change was intentional, or just missed due to not having any buildbots covering 
VS9.0-only builds.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29570] Windows Buildbot 2.7 is broken

[Jeremy Kloth]

Jeremy Kloth added the comment:

All builders have been reset and pass, with the exception of the VS9 one which 
is a different problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30357] test_thread.test_save_exception_state_on_error(): Unhandled exception in thread: AMD64 Debian root 2.7

[Grzegorz Grzywacz]

Changes by Grzegorz Grzywacz :


--
pull_requests: +1676

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30357] test_thread.test_save_exception_state_on_error(): Unhandled exception in thread: AMD64 Debian root 2.7

[Grzegorz Grzywacz]

Grzegorz Grzywacz added the comment:

Problem is with test 
test_thread.ThreadRunningTests.test_save_exception_state_on_error when other 
tests leave threads runnig. 

test_save_exception_state_on_error relay on thread._get_count(), if this value 
decrease test assume thread is finished with is not always correct (other 
threads finish - started by different test).

Fix is to make sure each test wait for all threads to finsh.

--
nosy: +grzgrzgrz3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30332] Errors that aren't actually defined are purple in the IDLE

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
type:  -> behavior
versions: +Python 3.6, Python 3.7 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12782] Multiple context expressions do not support parentheses for continuation across lines

[Ulrich Petri]

Ulrich Petri added the comment:

So this would basically be:

with ExitStack() as stack:
cm1 = stack.enter_context(a_long_name.with_a_long_method())
cm2 = stack.enter_context(another_long_variable.with_a_long_method())

Seems like a very non-obvious and inelegant solution...

--
nosy: +ulope

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29763] test_site failing on AppVeyor

[Zachary Ware]

Zachary Ware added the comment:


New changeset d48214f22c84f3d949457c854e263074994b6fa2 by Zachary Ware in 
branch 'master':
bpo-29763: Clean up _pth tests (GH-954)
https://github.com/python/cpython/commit/d48214f22c84f3d949457c854e263074994b6fa2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29570] Windows Buildbot 2.7 is broken

[Zachary Ware]

Zachary Ware added the comment:

Jeremy, have you recreated your checkouts since .gitattributes was committed?

--
nosy: +jkloth

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30360] getpass.getpass() does not accept stdin from an Expect script

[Josh Rosenberg]

Changes by Josh Rosenberg :


--
title: getpass.getpass() does not except stdin from an Expect script -> 
getpass.getpass() does not accept stdin from an Expect script

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30366] Backport tests for test.support to 2.7

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1675

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30366] Backport tests for test.support to 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Also backported new functions temp_dir() and python_is_optimized().
temp_cwd() now accepts None as a name (means using tempfile.mkdtemp).
check_syntax_error() now accepts arguments lineno and offset.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30366] Backport tests for test.support to 2.7

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Proposed patch backports tests for test.support to 2.7. test_support.py is an 
old name of test.support, so the test is renamed to test_test_support.py.

The patch also backports few useful functions in test.support for which tests 
are written.

--
assignee: serhiy.storchaka
components: Tests
messages: 293656
nosy: ezio.melotti, haypo, michael.foord, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Backport tests for test.support to 2.7
type: enhancement
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[Paul Moore]

Paul Moore added the comment:

> Since we don't AFAIK have any possibility of mixed virtual environments 
> ignoring SheBangs should meet the basic requirements of operating inside of a 
> venv.

If you want a script to use the current environment, then #!/usr/bin/env python 
does that. There's no need for special-casing a virtual environment.

In addition, the current behaviour mirrors the Unix behaviour, which is 
intentional (and beneficial).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30365] Backport warnings in ElementTree/cElementTree modules and fix bugs

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

getchildren() and getiterator() were deprecated in Python implementation for 
years (since updating to ElementTree 1.3 in Python 2.7 and 3.2). And they are 
documented as deprecated. Just C implementation didn't emit warnings. It may be 
too late to add unconditional deprecation warnings, but we can add them as Py3k 
warnings.

I'm going to remove these deprecated methods in 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30365] Backport warnings in ElementTree/cElementTree modules and fix bugs

[Stefan Behnel]

Stefan Behnel added the comment:

Looks good to me (didn't test it).

Note that getchildren() is not deprecated in lxml because it's actually the 
fastest way to build a list of the children. It's faster than list(element) 
because it avoids the Python (C-level) iteration overhead. However, that 
probably wouldn't be substantial enough to merit adding it if it wasn't there 
already. I'm ok with deprecating the method in ET. It's redundant there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30365] Backport warnings in ElementTree/cElementTree modules and fix bugs

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1674

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30365] Backport warnings in ElementTree/cElementTree modules and fix bugs

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Proposed patch add warnings in ElementTree and cElementTree modules (backports 
issue29204) when run Python with the -3 option and fixes several bugs in the 
etree package and its tests:

The deprecation warning about the doctype() method was emitted when use 
xml.etree.ElementTree.XMLParser. Now it emitted only when use a subclass of 
XMLParser with defined doctype() method, as intended.

In the test_bug_200708_close test an EchoTarget instance was incorrectly passed 
to XMLParser() as the html argument and silently ignored. Now it is passed as 
the target argument.

Tests no longer failed when use the -m option for running only selected test 
methods. Checking warnings now is more specific, warnings are expected only 
when use deprecated features.

--
components: Extension Modules, Library (Lib), XML
messages: 293652
nosy: benjamin.peterson, eli.bendersky, scoder, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Backport warnings in ElementTree/cElementTree modules and fix bugs
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Changes by Berker Peksag :


--
resolution:  -> fixed
stage: backport needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Berker Peksag added the comment:


New changeset f3291eeb90da2ddb37efea30dfc9406983b0f681 by Berker Peksag in 
branch '3.5':
bpo-30358: Document sort argument of profile.runctx() (GH-1566)
https://github.com/python/cpython/commit/f3291eeb90da2ddb37efea30dfc9406983b0f681


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Berker Peksag added the comment:


New changeset 61b6e5cf929a80bb44500fb1a212facf66d7f45c by Berker Peksag in 
branch '3.6':
bpo-30358: Document sort argument of profile.runctx() (GH-1566)
https://github.com/python/cpython/commit/61b6e5cf929a80bb44500fb1a212facf66d7f45c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30360] getpass.getpass() does not except stdin from an Expect script

[brent saner]

brent saner added the comment:

i have found a workaround- it seems that if i add a "sleep 2" in the expect 
script before and after the getpass() entry prompt points, it works. which 
doesn't make sense to me, considering the getpass() prompt seems to take just 
as long as the other prompts to render, but who knows.

i'm not sure if this is an invalid bug or not, i'd welcome ideas as to which 
it'd be. if invalid, feel free to close.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Changes by Berker Peksag :


--
pull_requests: +1673

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Changes by Berker Peksag :


--
pull_requests: +1672

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30364] Outdated function attribute to disable address sanitizer

[Xiang Zhang]

New submission from Xiang Zhang:

In obmalloc, __attribute__((no_address_safety_analysis)) is used to disable 
address sanitizer. But this attribute is now deprecated in both clang and GCC.

Related links:

https://clang.llvm.org/docs/AddressSanitizer.html#disabling-instrumentation-with-attribute-no-sanitize-address

https://gcc.gnu.org/onlinedocs/gcc-4.8.1/gcc/Function-Attributes.html

--
messages: 293648
nosy: xiang.zhang
priority: normal
severity: normal
status: open
title: Outdated function attribute to disable address sanitizer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30362] Launcher add list and list with paths options

[Steve Barnes]

Changes by Steve Barnes :


--
pull_requests: +1671

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30361] Docs example: converting mixed types to floating point

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
assignee: docs@python -> Mariatta
nosy: +Mariatta
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[Vinay Sajip]

Vinay Sajip added the comment:

> If the user is operating in a virtual environment they are ring fenced and, 
> presumably, don't wish any other versions of python to be used

I'm not sure that's true. I have a number of tools which rely on specific 
libraries and so I have created venvs for them. I then link the different tool 
executable scripts from e.g. ~/.virtualenvs/foo_env/bin to ~/bin. Each script, 
because its shebang references the interpreter in its venv, runs with the 
correct Python version and libraries available in that venv. Once set up, these 
scripts are treated by me as "black boxes" in the course of my normal workflow.

When I happen to activate some other venv in the course of my development work, 
I don't want all these tool scripts to stop working because they try to use the 
interpreter and libraries installed in that venv, rather than the venv that 
they live in.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28595] shlex.shlex should not augment wordchars

[Vinay Sajip]

Vinay Sajip added the comment:

> Do I need to create a pull request for this?

Yes, creating a PR is part of the current workflow since we moved to GitHub. 
Though that's not the reason why it's not been progressed - on my part, at 
least, it's due to not having had time to think through the implications of 
this change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30363] Backport warnings in the re module to 2.7

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1670

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30363] Backport warnings in the re module to 2.7

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The life of 2.7 is so long that our common practice of releasing one or two 
versions with warnings before removing a feature or changing the behavior in 
incompatible way doesn't work. People that will start porting to Python 3 in 
2020 will be surprised.

Proposed patch backports warnings in the re module to 2.7 in py3k mode. Some of 
these warnings already are converted to errors, but the most important warnings 
are warnings about changing semantic. '\u' and '\U' have different meaning in 
3.3+ (see issue3665). re.split() with potentially empty pattern now emits 
FutureWarning and will change the behavior in future versions (see issue22818).

Warnings about inline flags occurred not at the start of the regular expression 
(see issue22493 and issue30298) was not backported. Adding them requires 
changing the fnmatch module, non-trivially changing distutils, and likely will 
break third-party libraries.

Warnings are emitted only when Python is ran with the -3 option, and it is easy 
to fix regular expressions.

--
components: Extension Modules, Library (Lib), Regular Expressions
messages: 293645
nosy: benjamin.peterson, ezio.melotti, mrabarnett, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Backport warnings in the re module to 2.7
type: enhancement
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26467] Add async magic method support to unittest.mock.Mock

[Michał Bultrowicz]

Michał Bultrowicz added the comment:

I guess that it's worth noting that the same problem is being talked about in 
an issue on the asynctest library: 
https://github.com/Martiusweb/asynctest/issues/29

--
nosy: +butla

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30357] test_thread.test_save_exception_state_on_error(): Unhandled exception in thread: AMD64 Debian root 2.7

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I just encountered with this failure. It is random, I can reproduce it only 
when run test_thread repeatedly in parallel with running other tests (maybe 
needed high load factor).

$ ./python -m test -uall -F test_thread test_thread test_thread test_thread 
Run tests sequentially
0:00:00 [  1] test_thread
0:00:00 [  2] test_thread
Traceback (most recent call last):
  File "/home/serhiy/py/cpython2.7/Lib/test/test_thread.py", line 133, in task
raise SyntaxError
SyntaxError: None
test test_thread failed -- Traceback (most recent call last):
  File "/home/serhiy/py/cpython2.7/Lib/test/test_thread.py", line 150, in 
test_save_exception_state_on_error
self.assertIn("Traceback", stderr.getvalue())
AssertionError: 'Traceback' not found in 'Unhandled exception in thread started 
by \n'

1 test OK.
1 test failed:
test_thread

Total duration: 432 ms
Tests result: FAILURE
[50664 refs]

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30362] Launcher add list and list with paths options

[Steve Barnes]

New submission from Steve Barnes:

Currently the launcher will give an error message if a matching python is not 
found when using the -x.y-bits notation on the command line however the user 
has to know which python(s) they have available.

When some pythons have been installed all user and some specific user only the 
individual user may well not know which pythons they have available.

I would like to suggest adding to the command line -l and -L options for list 
and Long list options. Each option would populate the installed_pythons list 
and then

 -l print the list of usable -x.y-bits options currently available.
 -L print the list with the paths of the executable files.

In either case the launcher should terminate after printing the list.

This would be useful for deciding which python(s) to test with and for 
producing a list to be able to deploy critical updates to all installed pythons.

--
components: Windows
messages: 293642
nosy: Steve Barnes, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Launcher add list and list with paths options
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30361] Docs example: converting mixed types to floating point

[Mihai Borobocea]

New submission from Mihai Borobocea:

Section 3.1.1 in the Docs states:
"operators with mixed type operands convert the integer operand to floating 
point"
https://docs.python.org/3/tutorial/introduction.html

But gives an example using the division operator:
>>> 3 * 3.75 / 1.5
7.5
>>> 7.0 / 2
3.5

In Python 3, division always returns a float, e.g. when all operands are 
integers:
>>> 3 * 3 / 1
9.0
>>> 7 / 2
3.5

To illustrate that "operators with mixed type operands convert the integer 
operand to floating point" the example should not use division. For example:
>>> 3 * 3.75 - 1
10.25
>>> 7.0 + 2
9.0

--
assignee: docs@python
components: Documentation
messages: 293641
nosy: MihaiHimself, docs@python
priority: normal
severity: normal
status: open
title: Docs example: converting mixed types to floating point
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28686] py.exe ignored PATH when using python3 shebang

[Steve Barnes]

Steve Barnes added the comment:

If the user is operating in a virtual environment they are ring fenced and, 
presumably, don't wish any other versions of python to be used, (regardless of 
the SheBang in the files).

Since we don't AFAIK have any possibility of mixed virtual environments 
ignoring SheBangs should meet the basic requirements of operating inside of a 
venv.

How about a simple, but brutal approach, of if VIRTUAL_ENV is set in the 
current environment then always use the virtual environment python and launch 
with the -x option (ignore first line). This is the equivalent of using a 
version specifier as well as having a SheBang. This way the wrong python for 
the script might be executed but that is the responsibility of the person who 
set up the venv.

--
nosy: +Steve Barnes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Changes by Berker Peksag :


--
stage: patch review -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

Berker Peksag added the comment:


New changeset 99776296230ddd8429ebad2d07854b8c27ea10ab by Berker Peksag 
(csabella) in branch 'master':
bpo-30358: Document sort argument of profile.runctx() (GH-1566)
https://github.com/python/cpython/commit/99776296230ddd8429ebad2d07854b8c27ea10ab


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30340] Optimize out non-capturing groups

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30299] Display the bytecode when compiled a regular expression in debug mode

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30358] Profile documentation - include sort argument for runctx

[Berker Peksag]

New submission from Berker Peksag:

For future reference, this was also reported by Chris Cummins on docs@p.o at 
https://mail.python.org/pipermail/docs/2017-May/031420.html

--
nosy: +berker.peksag
stage:  -> patch review
versions: +Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29895] Distutils blows up with an incorrect pypirc, should be caught

[Berker Peksag]

Berker Peksag added the comment:

Thanks for the report, Tommy. I think the configparser exception is pretty 
descriptive here. You had a .pypirc file like the following:

[distutils]
index-servers = 
  pypi
  global

[pypi]
username:spam
password:eggs

You passed -r global and you got:

Traceback (most recent call last):
  ...
ConfigParser.NoSectionError: No section: 'global'

Which says you don't have a section named 'global' in your .pypirc.

There are three options here:

a) do nothing
b) catch configparser.NoSectionError and raise a distutils exception with 
better wording. Since we don't have DistutilsConfigError, I'm not sure which 
one is more appropriate here.
c) ignore configparser.NoSectionError if server name cannot be found in 
sections in PyPIRCCommand._read_pypirc(). IMO this is the worst option.

Here is a test case to demonstrate the problem described at the SO question.

--
keywords: +patch
nosy: +berker.peksag
versions: +Python 3.5, Python 3.6, Python 3.7
Added file: http://bugs.python.org/file46863/issue29895_test.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30299] Display the bytecode when compiled a regular expression in debug mode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 4ab6abfca4d6e444cca04821b24701cde6993f4e by Serhiy Storchaka in 
branch 'master':
bpo-30299: Display a bytecode when compile a regex in debug mode. (#1491)
https://github.com/python/cpython/commit/4ab6abfca4d6e444cca04821b24701cde6993f4e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29877] compileall hangs when accessing urandom even if number of workers is 1

[Berker Peksag]

Changes by Berker Peksag :


--
nosy: +Claudiu.Popa, brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com