[issue37424] subprocess.run timeout does not function if shell=True and capture_output=True

[haizaar]

New submission from haizaar :

Consider the following:

subprocess.run('sleep 10', shell=True, timeout=.1, capture_output=True)

It should raise after 0.1 seconds, but it does not - it waits 10 seconds till 
sleep finishes and only then raises "subprocess.TimeoutExpired: Command 'sleep 
10' timed out after 0.1 seconds"

Removing 'capture_output=True' or converting command string to list (and 
removing shell=True) makes it work.

I'm using Python 3.7.3 on Ubuntu 16.04. Reproduces on official docker Python 
3.7.3 image alpine3.8.

--
components: Library (Lib)
messages: 346712
nosy: haizaar
priority: normal
severity: normal
status: open
title: subprocess.run timeout does not function if shell=True and 
capture_output=True
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35389] Use gnu_get_libc_version() in platform.libc_ver()?

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a719c8f4bd3cc5b1d98700c15c4a818f4d5617a4 by Victor Stinner in 
branch 'master':
bpo-35389: platform.platform() calls libc_ver() without executable (GH-14418)
https://github.com/python/cpython/commit/a719c8f4bd3cc5b1d98700c15c4a818f4d5617a4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37424] subprocess.run timeout does not function if shell=True and capture_output=True

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

On mac this exits immediately with 0.1 seconds as timeout but reproducible on 
Ubuntu with master.

--
nosy: +gregory.p.smith, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37425] Using Pyinstaller with (Anaconda) Python 3.7.3 needs PyQt5 even if not imported in code

[Sameer Bobade]

New submission from Sameer Bobade :

I am writing a test code with Python and tested that it works on both Python 
3.7 and 3.7.3
The code uses tkinter, and does not use PyQT5
I use pyinstaller 3.4 I convert the python to exe.

With the same code and and same dependencies (including version number) - while 
converting code to exe using pyinstaller 3.4, with Python 3.7.3 - Pyinstaller 
asks for additional imports related to PyQt5 (and maybe more). This causes 
executable to expand in size with unwanted dependencies bundled in output.

Doing same thing with Python 3.7 removes the issue.
Not sure if this is Anaconda specific or general Python issue.

Both tests were done in virtual environment with Anaconda Python 3.7.3 as base.
Attached logs from pyinstaller for both versions.

Also reported to Anaconda - 
https://github.com/Anaconda-Platform/anaconda-project/issues/207

--
components: Windows
files: pyinstallerlog.zip
messages: 346715
nosy: Sameer Bobade, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Using Pyinstaller with (Anaconda) Python 3.7.3 needs PyQt5 even if not 
imported in code
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48440/pyinstallerlog.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Pavel Minaev]

Change by Pavel Minaev :


--
nosy: +int19h

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37415] Error build Python with Intel compiler

[Borja]

Borja  added the comment:

In pyconfig.h the variable HAVE_STD_ATOMIC appears like this:

/* Has stdatomic.h with atomic_int */
#define HAVE_STD_ATOMIC 1

I searched at  atomic_uintptr_t and found nothing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33926] test_gdb is skipped in builds since gdb is not installed as part of build script

[Jeroen Demeyer]

Change by Jeroen Demeyer :


--
keywords: +patch
pull_requests: +14233
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14395

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32926] Add public TestCase method/property to get result of current test

[Jeroen Demeyer]

Change by Jeroen Demeyer :


--
keywords: +patch
pull_requests: +14234
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14395

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36656] Please add race-free os.link and os.symlink wrapper / helper

[Tal Einat]

Change by Tal Einat :


--
components: +Library (Lib)
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32926] Add public TestCase method/property to get result of current test

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
pull_requests:  -14234

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32926] Add public TestCase method/property to get result of current test

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +ezio.melotti, michael.foord, rbcollins
stage: patch review -> 
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36656] Please add race-free os.link and os.symlink wrapper / helper

[Tal Einat]

Tal Einat  added the comment:

I think there is a general flaw with the suggested implementation approach.

An infinite loop as currently suggested seems dangerous to me.  For example, it 
could cause a program to stall indefinitely in some unforeseen edge-case.  I 
don't think this would be acceptable for something in the stdlib.

On the other hand, limiting this loop to a finite number of attempts would 
re-introduce the necessity of handling the case where creating the symlink 
fails. Users of the function would still need to deal with this case (or risk 
unhandled exceptions).

In either case, this wouldn't give users the simplicity and peace of mind that 
a truly atomic operation would.  ISTM that only using actually atomic 
operations, as provided by the underlying file system and/or OS, would allow us 
to provide that.

Given the above, I must say that I don't see this providing enough utility to 
justify inclusion in the stdlib, unless a better implementation approach can be 
found.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37424] subprocess.run timeout does not function if shell=True and capture_output=True

[Martin Panter]

Martin Panter  added the comment:

Same thing going on as in Issue 30154. The shell is probably spawning the 
“sleep” command as a child process (grandchild of Python), and waiting for it 
to exit. When Python times out, it will kill the shell process, leaving the 
grandchild as an orphan. The “sleep” process will still be running and probably 
holds the “stdout” and/or “stderr” pipes open, and Python will wait 
indefinitely to be sure it has captured all the output to those pipes.

Also see Issue 26534 proposes APIs to kill a process group rather than the 
single child process.

--
nosy: +martin.panter
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> subprocess.run with stderr connected to a pipe won't timeout 
when killing a never-ending shell commanad

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37366] Add an "onitem" callback parameter to shutil.rmtree()

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

I am not sure I understand the use case. Other than "unmounting a mount point" 
the other scenarios look unrealistic to me and also require a fd instead of a 
path. Also, if you want to unmount a path you can do so via *onerror* instead 
(unmount + delete). Pre-emptively checking if you have the permission to do 
something is sort of unpythonic and racy.

--
versions:  -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32926] Add public TestCase method/property to get result of current test

[Jeroen Demeyer]

Change by Jeroen Demeyer :


--
pull_requests: +14235
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14395

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32926] Add public TestCase method/property to get result of current test

[Jeroen Demeyer]

Change by Jeroen Demeyer :


--
pull_requests:  -14235

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Aldwin Pollefeyt]

Aldwin Pollefeyt  added the comment:

The second import actually doesn't happen. You need to reload it. This can be 
tested by putting print('loading threading') in threading.py.

Your first method-thread will still think it's main thread. So no idea if this 
is a bug or wrong use. 'import threading' should be one of the first lines in 
your main code/thread?


import _thread
import time
import importlib
barrier = 0

def method():
import threading  # Will make threading the wrong thread.
global barrier
print(threading.main_thread())
print(threading.current_thread())
barrier = 1

_thread.start_new_thread(method, ())

while barrier != 1:
time.sleep(.1)

import threading
importlib.reload(threading)
print(threading.main_thread())
print(threading.current_thread())

--
nosy: +aldwinaldwin

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37426] getpass.getpass not working with on windows when ctrl+v is used to enter the string

[Atul Bagga]

New submission from Atul Bagga :

Detailed issue filed here - https://github.com/microsoft/knack/issues/160

--
components: Library (Lib)
messages: 346721
nosy: Atul Bagga
priority: normal
severity: normal
status: open
title: getpass.getpass not working with on windows when ctrl+v  is used to 
enter the string
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37427] sorted() and list.sort() don't accept non-boolean objects with __bool__() as `reverse` parameter

[NODA, Kai]

New submission from NODA, Kai :

class X:
def __bool__(self):
return True

x = [1, 2, 3]
x.sort(reverse=X())
print(x)
print(sorted([1, 2, 3], reverse=X()))

TypeError: an integer is required (got type X)

We can always still do

x.sort(reverse=bool(X()))
print(sorted([1, 2, 3], reverse=bool(X(

but that isn't cool

--
components: Interpreter Core
messages: 346722
nosy: nodakai
priority: normal
severity: normal
status: open
title: sorted() and list.sort() don't accept non-boolean objects with 
__bool__() as `reverse` parameter
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Pavel Minaev]

Pavel Minaev  added the comment:

This is a bit tricky to explain... There's no easy way to achieve this effect 
"normally". It manifests due to the way some Python debuggers (specifically, 
pydevd and ptvsd - as used by PyCharm, PyDev, and VSCode) implement 
non-cooperative attaching to a running Python process by PID.

A TL;DR take is that those debuggers have to inject a new thread into a running 
process from the outside, and then run some Python code on that thread. There 
are OS APIs for such thread injection - e.g. CreateRemoteThread on Windows. 
There are various tricks that they then have to use to safely acquire GIL and 
invoke PyEval_InitThreads, but ultimately it comes down to running Python code.

That is the point where this can manifest. Basically, as soon as that injected 
code (i.e. the actual debugger) imports threading, things break. And advanced 
debuggers do need background threads for some functionality...

Here are the technical details - i.e. how thread injection happens exactly, and 
what kind of code it might run - if you're interested.
https://github.com/microsoft/ptvsd/issues/1542

I think that a similar problem can also occur in an embedded Python scenario 
with multithreading. Consider what happens if the hosted interpreter is 
initialized from the main thread of the host app - but some Python code is then 
run from the background thread, and that code happens to be the first in the 
process to import threading. Then that background thread becomes the "main 
thread" for threading, with the same results as described above.

The high-level problem, I think, is that there's an inconsistency between what 
Python itself considers "main thread" (i.e. main_thread in ceval.c, as set by 
PyEval_InitThreads), and what threading module considers "main thread" (i.e. 
_main_thread in threading.py). Logically, these should be in sync.

If PyEval_InitThreads is the source of truth, then the existing thread 
injection technique will "just work" as implemented already in all the 
aforementioned debuggers. It makes sure to invoke PyEval_InitThreads via 
Py_AddPendingCall, rather than directly from the background thread, precisely 
so that the interpreter doesn't get confused. 

Furthermore, on 3.7+, PyEval_InitThreads is already automatically invoked by 
Py_Initialize, and hence when used by python.exe, will mark the actual first 
thread in the process as the main thread. So, using it a the source of truth 
would guarantee that attach by thread injection works correctly in all 
non-embedded Python scenarios.

Apps hosting Python would still need to ensure that they always call 
Py_Initialize on what they want to be the main thread, as they already have to 
do; but they wouldn't need to worry about "import threading" anymore.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37427] sorted() and list.sort() don't accept non-boolean objects with __bool__() as `reverse` parameter

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Pavel Minaev]

Pavel Minaev  added the comment:

It's also possible to hit if using some native code that starts a background 
thread without going via threading, and runs Python code on that background 
thread. In that case, if that Python code then does "import threading", and 
threading hasn't been imported yet, then you have this same problem.

Here's a pure Python repro using ctypes on Win32:

#--
import sys, time
from ctypes import *

ThreadProc = WINFUNCTYPE(c_uint32, c_void_p)

@ThreadProc
def thread_proc(_):
import threading
print(threading.current_thread() is threading.main_thread())
return 0

assert "threading" not in sys.modules
#import threading  # uncomment to fix

windll.kernel32.CreateThread(None, c_size_t(0), thread_proc, None, c_uint32(0), 
None)
time.sleep(1)

assert "threading" in sys.modules
import threading
print(threading.current_thread() is threading.main_thread())
#--

Here's the output:

>py -3 main.py
True
False
Exception ignored in: 
Traceback (most recent call last):
  File "C:\Python\3.7-64\lib\threading.py", line 1276, in _shutdown
assert tlock.locked()
AssertionError:

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37426] getpass.getpass not working with on windows when ctrl+v is used to enter the string

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37428] SSLContext.post_handshake_auth implicitly enables cert validation

[Christian Heimes]

New submission from Christian Heimes :

Enabling TLS 1.3 post handshake auth also enables cert chain validation. 
OpenSSL documents SSL_VERIFY_POST_HANDSHAKE as ignored for client side. However 
tls_process_server_certificate in the client state machine code does not ignore 
the flag and checks for a correct cert chain.

see https://github.com/openssl/openssl/issues/9259 and 
https://github.com/openssl/openssl/blob/743694a6c29e5a6387819523fad5e3b7e613f1ee/ssl/statem/statem_clnt.c#L1899-L1918

--
assignee: christian.heimes
components: SSL
messages: 346725
nosy: christian.heimes
priority: high
severity: normal
status: open
title: SSLContext.post_handshake_auth implicitly enables cert validation
type: behavior
versions: Python 2.7, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Aldwin Pollefeyt]

Aldwin Pollefeyt  added the comment:

Understood. Thank you for the extra info. I'll read up on all the suggestions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37426] getpass.getpass not working with on windows when ctrl+v is used to enter the string

[Eryk Sun]

Eryk Sun  added the comment:

getpass() reads from the console via msvcrt.getwch(). This is a raw console 
read, so Ctrl+V is read as the SYN (0x16) control character. Only the following 
keys are handled specially by getpass: 

* Ctrl+C - raise KeyboardInterrupt
* Ctrl+H (backspace) - delete the previous character
* Ctrl+J (linefeed) and Ctrl+M (return) - return

getpass() behaves pretty much the same in a Linux terminal emulator, except 
Ctrl+H isn't backspace. 

Recent versions of the console host in Windows 10 have an option to support 
pasting via Ctrl+Shift+V, regardless of the input mode. This is pretty common 
in Unix terminal emulators as well. You can change this setting in the 
Properties->Options dialog for a particular window or shortcut, or the default 
setting in the Defaults->Options dialog. In the "HKCU\Console" registry key, 
the DWORD value name is "InterceptCopyPaste".

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33926] test_gdb is skipped in builds since gdb is not installed as part of build script

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 667eaffb4e5d03bf8129773f79649c3befaa5b1a by Inada Naoki (Jeroen 
Demeyer) in branch 'master':
bpo-33926: enable GDB tests on Travis CI (GH-14395)
https://github.com/python/cpython/commit/667eaffb4e5d03bf8129773f79649c3befaa5b1a


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33926] test_gdb is skipped in builds since gdb is not installed as part of build script

[miss-islington]

Change by miss-islington :


--
pull_requests: +14236
pull_request: https://github.com/python/cpython/pull/14420

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37428] SSLContext.post_handshake_auth implicitly enables cert validation

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch
pull_requests: +14237
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14421

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[miss-islington]

Change by miss-islington :


--
pull_requests: +14238
pull_request: https://github.com/python/cpython/pull/14422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33926] test_gdb is skipped in builds since gdb is not installed as part of build script

[miss-islington]

miss-islington  added the comment:


New changeset 7e9d706fd6e2f49d2bd829890c46868105767769 by Miss Islington (bot) 
in branch '3.8':
bpo-33926: enable GDB tests on Travis CI (GH-14395)
https://github.com/python/cpython/commit/7e9d706fd6e2f49d2bd829890c46868105767769


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37427] sorted() and list.sort() don't accept non-boolean objects with __bool__() as `reverse` parameter

[Mark Dickinson]

Mark Dickinson  added the comment:

I haven't looked at the source yet, but from experimentation I'm finding it 
rather hard to guess what the rules are for what works and what doesn't:

Python 3.7.3 (default, Mar 30 2019, 03:37:43) 
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import decimal, fractions, numpy
>>> sorted([1, 2, 3], reverse=0.5)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: integer argument expected, got float
>>> sorted([1, 2, 3], reverse=decimal.Decimal(0.5))
[1, 2, 3]
>>> sorted([1, 2, 3], reverse=fractions.Fraction(0.5))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: an integer is required (got type Fraction)
>>> sorted([1, 2, 3], reverse=numpy.int64(1))
[3, 2, 1]
>>> sorted([1, 2, 3], reverse=numpy.bool_(True))
[3, 2, 1]

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37427] sorted() and list.sort() don't accept non-boolean objects with __bool__() as `reverse` parameter

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37427] sorted() and list.sort() don't accept non-boolean objects with __bool__() as `reverse` parameter

[Mark Dickinson]

Mark Dickinson  added the comment:

> I'm finding it rather hard to guess what the rules are for what works and 
> what doesn't

So now that I've looked at the source: anything with an `__int__` method works, 
except that `float` instances are explicitly excluded. So this explains for 
example:

>>> sorted([1, 2, 3], reverse=numpy.float64(0.5))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: integer argument expected, got float
>>> sorted([1, 2, 3], reverse=numpy.float32(0.5))
[1, 2, 3]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37429] Python hangs on fork when a logger is in use in a background thread

[Ryan Petrello]

New submission from Ryan Petrello :

It's possible to cause Python child processes to encountered a deadlock and 
hang on fork by:

1) Spawning one or more threads in a parent process
2) Have one of those threads emit log lines
3) Have the main thread use os.fork()
4) In the child, log immediately

The attached Python script can be used to reproduce this issue in Python 2.7.15 
and Python 3.6.5

--
components: Library (Lib)
files: hang.py
messages: 346732
nosy: ryan.petrello
priority: normal
severity: normal
status: open
title: Python hangs on fork when a logger is in use in a background thread
versions: Python 2.7, Python 3.6
Added file: https://bugs.python.org/file48441/hang.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37429] Python hangs on fork when a logger is in use in a background thread

[Ryan Petrello]

Ryan Petrello  added the comment:

This issue seems _very_ similar to https://bugs.python.org/issue6721, but I 
wasn't able to encounter the exact stack I've been seeing on my end.  
Specifically, when I run the reproducer script in this issue (hang.py) for a 
few minutes, hung child processes start to build up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests: +14239
pull_request: https://github.com/python/cpython/pull/14423

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37429] Python hangs on fork when a logger is in use in a background thread

[Ryan Petrello]

Ryan Petrello  added the comment:

Here's the stack of one the hung child processes under py2.7:

(gdb) py-bt
#4 Waiting for a lock (e.g. GIL)
#5 Waiting for a lock (e.g. GIL)
#7 Frame 0x7f346a925430, for file /usr/lib64/python2.7/Queue.py, line 118, in 
put (self=, maxsize=0, all_tasks_done=<_Condition(_Verbose__verbose=False, 
_Condition__lock=, acquire=, 
_Condition__waiters=[], release=) at remote 0x7f3467df4150>, mutex=, not_full=<_Condition(_Verbose__verbose=False, 
_Condition__lock=, acquire=, 
_Condition__waiters=[], release=) at remote 0x7f3467df4110>, 
not_empty=<_Condition(_Verbose__verbose=False, _Condition__lock=, acquire=, _shutdown=False, _max_workers=2, 
_thread_name_prefix='Thread---Type  to continue, or q  to 
quit---
PoolExecutor-0', _threads=set([<...>, <...>]), _work_queue=<...>) at remote 
0x7f3467df0f90>, fn=, 
args=(, process=1155, 
processName='Process-74', args=(), module='hang', filename='hang.py', 
levelno=20, exc_text=None, pathname='/tmp/hang.py', lineno=25, msg='exiting', 
exc_info=None, funcName='exit', relativeCreated=, 
levelname='INFO', msecs=) at remote 
0x7f346a972490>,), kwargs={}, f=, _shutdown=False, _max_workers=2, 
_thread_name_prefix='ThreadPoolExecutor-0', _threads=set([<...>, <...>]), 
_work_queue=<...>) at remote 0x7f3467df0f90>) at remote 0x7f3467df0f50>, 
record=, process=1155, 
processName='Process-74', args=(), module='hang', filename='hang.py', 
levelno=20, exc_text=None, pathname='/tmp/hang.py', lineno=25, msg='exiting', 
exc_info=None, funcName='exit', relativeCreated= to continue, 
or q  to quit---
loat at remote 0x1a7d568>, levelname='INFO', msecs=) 
at remote 0x7f346a972490>)
self.executor.submit(self.send, record)
#19 Frame 0x7f346a925240, for file /usr/lib64/python2.7/logging/__init__.py, 
line 1318, in callHandlers (self=, _shutdown=False, _max_workers=2, 
_thread_name_prefix='ThreadPoolExecutor-0', _threads=set([<...>, <...>]), 
_work_queue=<...>) at remote 0x7f3467df0f90>) at remote 0x7f3467df0f50>], 
level=10, disabled=0, propagate=1, filters=[]) at remote 0x7f3467de1a90>, 
record=, process=1155, 
processName='Process-74', args=(), module='hang', filename='hang.py', 
levelno=20, exc_text=None, pathname='/tmp/hang.py', lineno=25, msg='exiting', 
exc_info=None, funcName='exit', relativeCreated=, 
levelname='INFO', msecs=) at remote 0x7f346a972490>, 
c=<...>, found=1, hdlr=<...>)
hdlr.handle(record)
#22 Frame 0x7f3467df13e0, for file /usr/lib64/python2.7/logging/__init__.py, 
line 1278, in handle (self=, _shut---Type  to continue, or q  to 
quit---
down=False, _max_workers=2, _thread_name_prefix='ThreadPoolExecutor-0', 
_threads=set([<...>, <...>]), _work_queue=<...>) at remote 0x7f3467df0f90>) at 
remote 0x7f3467df0f50>], level=10, disabled=0, propagate=1, filters=[]) at 
remote 0x7f3467de1a90>, record=, 
process=1155, processName='Process-74', args=(), module='hang', 
filename='hang.py', levelno=20, exc_text=None, pathname='/tmp/hang.py', 
lineno=25, msg='exiting', exc_info=None, funcName='exit', 
relativeCreated=, levelname='INFO', msecs=) at remote 0x7f346a972490>)
self.callHandlers(record)
#25 Frame 0x1ae60e0, for file /usr/lib64/python2.7/logging/__init__.py, line 
1268, in _log (self=, _shutdown=False, _max_workers=2, 
_thread_name_prefix='ThreadPoolExecutor-0', _threads=set([<...>, <...>]), 
_work_queue=<...>) at remote 0x7f3467df0f90>) at remote 0x7f3467df0f50>], 
level=10, disabled=0, propagate=1, filters=[]) at remote 0x7f3467de1a90>, 
level=20, msg='exiting', args=(), exc_info=None, extra=None, fn='/tmp/hang.py', 
lno=25, func='exit', record=, process=1155, 
processName='Process-74', args=(...), ---Type  to continue, or q 
 to quit---

Here's the similar stack under py3.6:

(gdb) bt
#0  0x7f208166cadb in futex_abstimed_wait (cancel=true, private=,
abstime=0x0, expected=0, futex=0x88bbf0)
at ../nptl/sysdeps/unix/sysv/linux/sem_waitcommon.c:43
#1  do_futex_wait (sem=sem@entry=0x88bbf0, abstime=0x0)
at ../nptl/sysdeps/unix/sysv/linux/sem_waitcommon.c:223
#2  0x7f208166cb6f in __new_sem_wait_slow (sem=0x88bbf0, abstime=0x0)
at ../nptl/sysdeps/unix/sysv/linux/sem_waitcommon.c:292
#3  0x7f208166cc0b in __new_sem_wait (sem=)
at ../nptl/sysdeps/unix/sysv/linux/sem_wait.c:28
#4  0x7f2081a05a80 in PyThread_acquire_lock_timed (lock=0x88bbf0,
microseconds=microseconds@entry=-1, intr_flag=intr_flag@entry=0)
at /usr/src/debug/Python-3.6.8/Python/thread_pthread.h:386
#5  0x7f2081a05c34 in PyThread_acquire_lock (lock=,
waitflag=waitflag@entry=1)
at /usr/src/debug/Python-3.6.8/Python/thread_pthread.h:595
#6  0x7f2081918797 in _enter_buffered_busy (self=self@entry=0x7f2081e29b48)
at /usr/src/debug/Python-3.6.8/Modules/_io/bufferedio.c:292
#7  0x7f2081a1f131 in buffered_flush (self=0x7f2081e29b48, args=)
at /usr/src/debug/Python-3.6.8/Modules/_io/bufferedio.c:852
---Type  to continue, or q  to quit---
#8  0x7f2081962dd0 in _PyCFunction_FastCallDict (
func_obj=fun

[issue37424] subprocess.run timeout does not function if shell=True and capture_output=True

[haizaar]

haizaar  added the comment:

Thanks for looking at it.

My original code had "tar" running, which is a child of the shell as well... I 
assume running exec in the shell may help somewhat, but not a cure obviously.

I'm all for killing the process group. "Run something and get it's about" 
should be simple enough without requiring a programmer to know all POSIX 
process semantics.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26534] subprocess.check_output with shell=True ignores the timeout

[haizaar]

Change by haizaar :


--
nosy: +haizaar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30154] subprocess.run with stderr connected to a pipe won't timeout when killing a never-ending shell commanad

[haizaar]

Change by haizaar :


--
nosy: +haizaar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37429] Python hangs on fork when a logger is in use in a background thread

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

See also https://bugs.python.org/issue36533

--
nosy: +vinay.sajip, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5115] Extend subprocess.kill to be able to kill process groups

[haizaar]

Change by haizaar :


--
nosy: +haizaar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37429] Python hangs on fork when a logger is in use in a background thread

[Ryan Petrello]

Ryan Petrello  added the comment:

Also, for what it's worth, I'm *not* able to reproduce this hang in Python 
3.7.0.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37429] Python hangs on fork when a logger is in use in a background thread

[Ryan Petrello]

Ryan Petrello  added the comment:

Actually, I think I *am* seeing this on 3.7:

(gdb) bt
#0  0x7f130d530adb in futex_abstimed_wait (cancel=true, private=, abstime=0x0, expected=0,
futex=0x142c6d0) at ../nptl/sysdeps/unix/sysv/linux/sem_waitcommon.c:43
#1  do_futex_wait (sem=sem@entry=0x142c6d0, abstime=0x0) at 
../nptl/sysdeps/unix/sysv/linux/sem_waitcommon.c:223
#2  0x7f130d530b6f in __new_sem_wait_slow (sem=0x142c6d0, abstime=0x0)
at ../nptl/sysdeps/unix/sysv/linux/sem_waitcommon.c:292
#3  0x7f130d530c0b in __new_sem_wait (sem=) at 
../nptl/sysdeps/unix/sysv/linux/sem_wait.c:28
#4  0x00535baa in PyThread_acquire_lock_timed (lock=0x142c6d0, 
microseconds=microseconds@entry=-1,
intr_flag=intr_flag@entry=0) at Python/thread_pthread.h:340
#5  0x00535c24 in PyThread_acquire_lock (lock=, 
waitflag=waitflag@entry=1)
at Python/thread_pthread.h:576
#6  0x0058949a in _enter_buffered_busy (self=0x7f130ca0fbf8) at 
./Modules/_io/bufferedio.c:282
#7  buffered_flush (self=0x7f130ca0fbf8, args=) at 
./Modules/_io/bufferedio.c:835
#8  0x0043aae8 in _PyMethodDef_RawFastCallDict (kwargs=, 
nargs=0, args=,
self=0x7f130ca0fbf8, method=0x8eeda0 ) at 
Objects/call.c:482
#9  _PyCFunction_FastCallDict (func=0x7f130aeb3438, args=, 
nargs=0, kwargs=)
at Objects/call.c:582

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37415] Error build Python with Intel compiler: doesn't provide atomic_uintptr_t

[STINNER Victor]

STINNER Victor  added the comment:

> I searched at  atomic_uintptr_t and found nothing.

That's not good. I expectd std to stand for standard. I expected that 
atomic_uintptr_t would always be available on .

GCC defines the type as:

typedef _Atomic __UINTPTR_TYPE__ atomic_uintptr_t;

Maybe you should try:

typedef struct _Py_atomic_address {
_Atomic Py_uintptr_t _value;
} _Py_atomic_address;

--
title: Error build Python with Intel compiler -> Error build Python with Intel 
compiler:  doesn't provide atomic_uintptr_t

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36889] Merge StreamWriter and StreamReader into just asyncio.Stream

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33926] test_gdb is skipped in builds since gdb is not installed as part of build script

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

Feel free to close...

--
nosy: +jdemeyer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33926] test_gdb is skipped in builds since gdb is not installed as part of build script

[STINNER Victor]

STINNER Victor  added the comment:

> Feel free to close...

What about 2.7 and 3.7 branches?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37427] sorted() and list.sort() don't accept non-boolean objects with __bool__() as `reverse` parameter

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See issue15999.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37412] os.getcwdb() doesn't implement PEP 528 (UTF-8) on Windows

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +14240
pull_request: https://github.com/python/cpython/pull/14424

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37412] os.getcwdb() doesn't implement PEP 528 (UTF-8) on Windows

[STINNER Victor]

STINNER Victor  added the comment:

> This update breaks long-path support in Windows.

Oops.

This is the risk of not testing a feature :-) My PR added very basic tests for 
getcwd() and getcwdb(). I wrote PR 14424 to not only fix the integer overflow 
check, but also to add a new unit tests for "long path".

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37425] Using Pyinstaller with (Anaconda) Python 3.7.3 needs PyQt5 even if not imported in code

[Steve Dower]

Steve Dower  added the comment:

This looks like because you have Qt installed in one environment and not the 
other, so when matplotlib imports it pyinstaller decides to include it.

You'll have to report this to pyinstaller. It's nothing to do with core Python.

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37430] range is not a built-in function

[bob gailer]

New submission from bob gailer :

In section 8.3 The for statement there is a reference to range as a built-in 
function. That was true in python 2. Now range is a built-in type.

--
assignee: docs@python
components: Documentation
messages: 346745
nosy: bgailer, docs@python
priority: normal
severity: normal
status: open
title: range is not a built-in function
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Eric Snow]

Eric Snow  added the comment:

Note that in Python 3.7 we consolidated a bunch of the global runtime state.  
The "main_thread" static in ceval.c moved to the internal struct 
_PyRuntimestate and in 3.7 it is accessible as 
_Runtime.ceval.pending.main_thread (but only if you build with Py_BUILD_CORE, 
which extensions shouldn't).  In 3.8 it was moved to the top of the struct as 
_PyRuntimeState.main_thread.  All that said, that thread ID is still not 
exposed directly in Python.

So perhaps it would make sense to add something like sys.get_main_thread_id, 
which Lib/threading.py could then use (rather than assuming the current (during 
import) thread is the main thread).

Unfortunately, this wouldn't help with anything earlier than 3.9.  Currently 
3.8 is already in beta1, where we apply a feature freeze, so it's barely too 
late for 3.8.  I suppose you could ask for a special exception from the release 
manager, given the change would be relatively small, with real benefits, but 
don't expect it. :)

--
nosy: +brett.cannon, eric.snow
versions: +Python 3.8, Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Eric Snow]

Eric Snow  added the comment:

FWIW, this might also have implications for thread shutdown during 
runtime/interpreter finalization.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35121] Cookie domain check returns incorrect results

[Riccardo Schirone]

Riccardo Schirone  added the comment:

Did anybody request a CVE for this issue? I think it deserves one as it is a 
security issue and it may leak cookies to wrong domains. Does anybody have 
anything against assigning a CVE to this issue? If not, I would try to get one 
from MITRE.

--
nosy: +rschiron

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35121] Cookie domain check returns incorrect results

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I also reported it to secur...@python.org . Please check with them too to see 
if there is a CVE request already made. Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37416] If threading is not imported from the main thread it sees the wrong thread as the main thread.

[Pavel Minaev]

Pavel Minaev  added the comment:

Debuggers will have to work around this in past Python versions that they 
support (which still includes Python 2 for pretty much all of them), so this is 
solely about resolving the inconsistency for the future. No point rushing it 
for 3.8 specifically.

(The most likely immediate workaround will be that, instead of invoking 
PyEval_InitThreads on the main thread via Py_AddPendingCall, we will simply use 
the same facility to exec "import threading" on the main thread.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37420] os.sched_setaffinity does not handle errors during iteration.

[miss-islington]

Change by miss-islington :


--
pull_requests: +14241
pull_request: https://github.com/python/cpython/pull/14425

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37420] os.sched_setaffinity does not handle errors during iteration.

[Benjamin Peterson]

Benjamin Peterson  added the comment:


New changeset 45a30af109f69a81576b87ea775863ba12d55316 by Benjamin Peterson 
(Brandt Bucher) in branch 'master':
closes bpo-37420: Handle errors during iteration in os.sched_setaffinity. 
(GH-14414)
https://github.com/python/cpython/commit/45a30af109f69a81576b87ea775863ba12d55316


--
nosy: +benjamin.peterson
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37420] os.sched_setaffinity does not handle errors during iteration.

[miss-islington]

Change by miss-islington :


--
pull_requests: +14242
pull_request: https://github.com/python/cpython/pull/14426

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37425] Using Pyinstaller with (Anaconda) Python 3.7.3 needs PyQt5 even if not imported in code

[Sameer Bobade]

Sameer Bobade  added the comment:

PyQt5 was installed when trying to make exe with Python 3.7.3 because without 
it pyinstaller will fail to produce output.

Added log without PyQt5 & Python 3.7.3 which fails to produce output.
Maybe this is issue of Matplotlib then? Not sure if versions are changed 
between the distribution.

--
status: closed -> open
Added file: https://bugs.python.org/file48442/withpy3.7.3 -without PyQt5.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37430] range is not a built-in function

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I presume you are referring to 8.3 of the language reference
https://docs.python.org/3/reference/compound_stmts.html#the-for-statement
which has "the built-in function range()".  Types/classes *are* functions, 
which is why dist, list, range, etc are listed in "Built-in Functions" in the 
library reference.  But I agree that the more specific term should be used.

To me, the more severe problem is with the complete sentence.

"Hint: the built-in function range() returns an iterator of integers suitable 
to emulate the effect of Pascal’s for i := a to b do; e.g., list(range(3)) 
returns the list [0, 1, 2]."

The now obsolete definition of Python in terms of the now obscure Pascal should 
be deleted here and anywhere else such remains.  I think the whole sentence 
should just be deleted.  The whole paragraph and example could otherwise be 
improved.

--
nosy: +terry.reedy
versions: +Python 3.9 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37420] os.sched_setaffinity does not handle errors during iteration.

[miss-islington]

miss-islington  added the comment:


New changeset 65e187239a6feb09840ed9c2ebee51ed151e690d by Miss Islington (bot) 
in branch '3.7':
closes bpo-37420: Handle errors during iteration in os.sched_setaffinity. 
(GH-14414)
https://github.com/python/cpython/commit/65e187239a6feb09840ed9c2ebee51ed151e690d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37422] Documentation on the change of __path__ in Python 3

[Jim Li]

Jim Li  added the comment:

Hi Eric,

Sorry for the late reply. I think I did not accurately describe the issue at 
all. As a minimal example, set up two virtual environments, one from 2.7.x, one 
from 3.7.2.

When you are in the virtual environment, do
- pip install protobuf==3.3.0
- python (to go into compiler)
- import google
- print(type(google.__path__))

The 2.7.x would tell you it's a 
The 3.7.2 would tell you it's a 


Since it's not a list, some behaviours are gone, e.g., insert. I'm not trying 
to manipulate protobuf package's __path__, but trying to upgrade a legacy 
codebase, the legacy codebase does something like:

module.insert(0, 'some/path'), which breaks under python3, because it's not a 
list anymore, but a .


I believe the problem comes from the 'top level entry point'. For example, when 
you have 2 separate packages but they were organized under the same namespace.

E.g., in their setup.py

Package A:

packages=[
"orange",
"orange.schemas",
],
namespace_packages=["orange"]

Package B:
packages=[
"orange",
"orange.server",
],
namespace_packages=["orange"]

The problem comes from when you try to do `import orange`, and do operations 
with orange.__path__ here. In Python 2.7, it's a list; in Python 3.7, it's a 


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37425] Using Pyinstaller with (Anaconda) Python 3.7.3 needs PyQt5 even if not imported in code

[Steve Dower]

Steve Dower  added the comment:

In any case, it's definitely pyinstaller and not CPython. Please report it on 
their issue tracker.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37420] os.sched_setaffinity does not handle errors during iteration.

[miss-islington]

miss-islington  added the comment:


New changeset 6fbed5350c786ab88868925d67f59e19d0ab841c by Miss Islington (bot) 
in branch '3.8':
closes bpo-37420: Handle errors during iteration in os.sched_setaffinity. 
(GH-14414)
https://github.com/python/cpython/commit/6fbed5350c786ab88868925d67f59e19d0ab841c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37431] test_asyncio: test_start_tls_server_1() failed on ARMv7 Debian buster 3.7

[STINNER Victor]

New submission from STINNER Victor :

The test failed but then passed when run again.

See also bpo-35998 "test_asyncio: test_start_tls_server_1() TimeoutError on 
Fedora 29" and bpo-35031 "test_asyncio test_start_tls_server_1 fails in AMD64 
FreeBSD CURRENT buildbots"

https://buildbot.python.org/all/#/builders/177/builds/241
== Tests result: FAILURE then SUCCESS ==

1 re-run test:
test_asyncio


==
ERROR: test_start_tls_server_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests)
--
Traceback (most recent call last):
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/test_sslproto.py",
 line 583, in test_start_tls_server_1
self.loop.run_until_complete(run_main())
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/asyncio/base_events.py",
 line 577, in run_until_complete
raise RuntimeError('Event loop stopped before Future completed.')
RuntimeError: Event loop stopped before Future completed.



test_start_tls_server_1 (test.test_asyncio.test_sslproto.ProactorStartTLSTests) 
... skipped 'Windows only'
test_start_tls_slow_client_cancel 
(test.test_asyncio.test_sslproto.ProactorStartTLSTests) ... skipped 'Windows 
only'
test_start_tls_wrong_args 
(test.test_asyncio.test_sslproto.ProactorStartTLSTests) ... skipped 'Windows 
only'
test_buf_feed_data (test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... 
ok
test_create_connection_memory_leak 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_create_connection_ssl_failed_certificate 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_create_connection_ssl_slow_handshake 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_handshake_timeout (test.test_asyncio.test_sslproto.SelectorStartTLSTests) 
... ok
test_start_tls_client_buf_proto_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_start_tls_client_corrupted_ssl 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_start_tls_client_reg_proto_1 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_start_tls_server_1 (test.test_asyncio.test_sslproto.SelectorStartTLSTests) 
... Exception in thread test-client:
Traceback (most recent call last):
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/functional.py",
 line 198, in run
self._prog(TestSocketWrapper(self._sock))
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/test_sslproto.py",
 line 573, in 
with self.tcp_client(lambda sock: client(sock, addr),
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/test_sslproto.py",
 line 518, in client
answer = sock.recv_all(len(ANSWER))
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/functional.py",
 line 138, in recv_all
data = self.recv(n - len(buf))
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/ssl.py", line 
1045, in recv
return self.read(buflen)
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/ssl.py", line 
920, in read
return self._sslobj.read(len)
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2488)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/threading.py", 
line 926, in _bootstrap_inner
self.run()
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/functional.py",
 line 200, in run
self._test._abort_socket_test(ex)
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/test/test_asyncio/functional.py",
 line 122, in _abort_socket_test
self.fail(ex)
  File 
"/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/unittest/case.py",
 line 693, in fail
raise self.failureException(msg)
AssertionError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2488)

ERROR
test_start_tls_slow_client_cancel 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... 
/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/threading.py:960:
 ResourceWarning: unclosed 
  del exc_type, exc_value, exc_tb
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ok
test_start_tls_wrong_args 
(test.test_asyncio.test_sslproto.SelectorStartTLSTests) ... ok
test_close_during_handshake 
(test.test_asyncio.test_sslproto.SslProtoHandshakeTests) ... 
/ssd/buildbot/buildarea/3.7.gps-ubuntu-exynos5-armv7l/build/Lib/unittest/mock.py:953:
 ResourceWarning: unclosed 
  self.side_effect = side_effect
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Unhandled error in exception handler
context: {'mess

[issue37422] Documentation on the change of __path__ in Python 3

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 21cfae107e410bf4b0ab3c142ca4449bc33290f5 by Inada Naoki in branch 
'master':
bpo-30345: travis: use -Og with --with-pydebug (GH-14423)
https://github.com/python/cpython/commit/21cfae107e410bf4b0ab3c142ca4449bc33290f5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO

[miss-islington]

Change by miss-islington :


--
pull_requests: +14243
pull_request: https://github.com/python/cpython/pull/14427

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO

[Inada Naoki]

Inada Naoki  added the comment:

I think test_gdb is useful for 3.8 branch because some PEP 590 relating changes 
will be backported to it.
But I don't know how test_gdb is useful for older branches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37369] Issue with pip in venv on Powershell in Windows

[Steve Dower]

Steve Dower  added the comment:

So I have a fix for 3.9 for this, I believe (involves making 
sys.base_executable configurable via PyConfig and overriding it and 
config->home when we know we're inside an app container).

For 3.8 I can apply the same fix and make sys.base_executable public, or keep 
it as sys._base_executable. Either way, it'll show up in PyConfig. (Victor - 
thoughts here? Should be a non-breaking change, right?)

For 3.7, I'm going to have to come up with a different fix, and possibly expose 
some new private API to be able to pass through the correct executable path.


Fundamentally, the problem is that Windows is going to start enforcing some 
ACLs that were previously not being enforced. In theory, when an application is 
deployed in an app package (as required through the Store), you are not 
supposed to be able to launch the executable directly. Instead, you should go 
through your current user's apps directory 
(C:\Users\name\AppData\Local\Microsoft\WindowsApps\, which 
contains only symlinks to the actual executables).

This has a very good advantage that the package family name does not change 
with new versions, while the full install path does. Currently, a 
behind-the-scenes update of Python will actually break environments when their 
"home" path is no longer correct.

However, it does mean that we need to start reporting a sys.executable that is 
not inside sys.prefix, and I'm not sure whether that is a good idea.

Looking at venv (+Vinay for confirmation here), it seems to treat "home" as 
"the directory containing the original Python binary" rather than "the 
directory containing the Python install". So in theory (and in my simple 
testing), using the sys.executable-not-in-prefix here is fine. But is that 
intentional?

--
nosy: +vinay.sajip, vstinner
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37369] Issue with pip in venv on Powershell in Windows

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +14244
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14428

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30345] test_gdb fails on Python 3.6 when built with LTO+PGO

[miss-islington]

miss-islington  added the comment:


New changeset 60f24b23bf6ac485d195bb904635bdc3fe646b07 by Miss Islington (bot) 
in branch '3.8':
bpo-30345: travis: use -Og with --with-pydebug (GH-14423)
https://github.com/python/cpython/commit/60f24b23bf6ac485d195bb904635bdc3fe646b07


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37390] Generate table of audit events for docs

[miss-islington]

Change by miss-islington :


--
pull_requests: +14245
pull_request: https://github.com/python/cpython/pull/14429

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37390] Generate table of audit events for docs

[Steve Dower]

Steve Dower  added the comment:


New changeset 44f91c388a6f4da9ed3300df32ca290b8aa104ea by Steve Dower in branch 
'master':
bpo-37390: Add audit event table to documentations (GH-14406)
https://github.com/python/cpython/commit/44f91c388a6f4da9ed3300df32ca290b8aa104ea


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37432] Fix a small param type allocation.rst

[hai shi]

Change by hai shi :


--
assignee: docs@python
components: Documentation
nosy: docs@python, shihai1991
priority: normal
severity: normal
status: open
title: Fix a small param type allocation.rst
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37432] Fix a small param type in allocation.rst

[hai shi]

Change by hai shi :


--
title: Fix a small param type allocation.rst -> Fix a small param type in 
allocation.rst

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37432] Fix a small param type in allocation.rst

[SilentGhost]

New submission from SilentGhost :

Could you please elaborate on what exactly needs fixing?

--
nosy: +SilentGhost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37432] Fix a small param type in allocation.rst

[hai shi]

Change by hai shi :


--
keywords: +patch
pull_requests: +14246
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14430

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37432] Fix a small param type in allocation.rst

[hai shi]

hai shi  added the comment:

Due to https://github.com/python/cpython/blob/master/Include/objimpl.h#L102,
`void PyObject_Del(PyObject *op)` should be `void PyObject_Del(void *op)`

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37390] Generate table of audit events for docs

[miss-islington]

miss-islington  added the comment:


New changeset 4fee28aa42dbac7f08a6d2829546b7d8e848034d by Miss Islington (bot) 
in branch '3.8':
bpo-37390: Add audit event table to documentations (GH-14406)
https://github.com/python/cpython/commit/4fee28aa42dbac7f08a6d2829546b7d8e848034d


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37390] Generate table of audit events for docs

[Steve Dower]

Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37433] syntax error in f-string in multiline string produces ~40k spaces of output

[Anthony Sottile]

New submission from Anthony Sottile :

For instance this file:

foo = f"""{}
foo"""

$ python3.8 --version --version
Python 3.8.0b1 (default, Jun  6 2019, 03:44:52) 
[GCC 7.4.0]

$ python3.8 t.py | wc -c
  File "t.py", line 1


<>

   ^
SyntaxError: f-string: empty expression not allowed


$ python3.8 t.py |& wc -c
49134

--
components: Interpreter Core
messages: 346767
nosy: Anthony Sottile
priority: normal
severity: normal
status: open
title: syntax error in f-string in multiline string produces ~40k spaces of 
output
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37434] Segfault in typeobject.c at _PyObject_GC_UNTRACK(type)

[Tim Lusk]

New submission from Tim Lusk :

I'm working on porting an application from Python 3.6.8 to 3.7.3 and am running 
into this segfault.
I'm not able to pin point exactly what Python code is causing this as it's a 
very large enterprise application
and I can't seem to pinpoint exactly where this is coming from.

The application consistently crashes on start every time with this stack trace.

Program received signal SIGSEGV, Segmentation fault.
0x77922ae5 in type_dealloc (type=0x7fffed185480) at 
/usr/src/debug/Python-3.7.3/Objects/typeobject.c:3283
3283_PyObject_GC_UNTRACK(type);
(gdb) bt
#0  0x77922ae5 in type_dealloc (type=0x7fffed185480) at 
/usr/src/debug/Python-3.7.3/Objects/typeobject.c:3283
#1  0x778ee721 in free_keys_object (keys=0x7fffee258bc0) at 
/usr/src/debug/Python-3.7.3/Objects/dictobject.c:559
#2  0x778f20af in dict_dealloc (mp=0x7fffee209c60) at 
/usr/src/debug/Python-3.7.3/Objects/dictobject.c:1913
#3  0x77904d15 in module_clear (m=0x7fffee211048) at 
/usr/src/debug/Python-3.7.3/Objects/moduleobject.c:749
#4  0x77a3aeda in delete_garbage (collectable=0x7fff7420, 
old=0x77dd75f0 <_PyRuntime+400>) at 
/usr/src/debug/Python-3.7.3/Modules/gcmodule.c:769
#5  0x77a3b41d in collect (generation=2, n_collected=0x7fff7488, 
n_uncollectable=0x7fff7490, nofail=0)
at /usr/src/debug/Python-3.7.3/Modules/gcmodule.c:924
#6  0x77a3b845 in collect_with_callback (generation=2) at 
/usr/src/debug/Python-3.7.3/Modules/gcmodule.c:1036
#7  0x77a3bcef in gc_collect_impl (module=0x7615d9f8, generation=2) 
at /usr/src/debug/Python-3.7.3/Modules/gcmodule.c:1138
#8  0x77a3ba13 in gc_collect (module=0x7615d9f8, args=0xcc6cd0, 
nargs=0, kwnames=0x0) at 
/usr/src/debug/Python-3.7.3/Modules/clinic/gcmodule.c.h:100
#9  0x778b48e5 in _PyMethodDef_RawFastCallKeywords 
(method=0x77d8b800 , self=0x7615d9f8, args=0xcc6cd0, 
nargs=0, kwnames=0x0)
at /usr/src/debug/Python-3.7.3/Objects/call.c:656
#10 0x778b4bab in _PyCFunction_FastCallKeywords (func=0x7615ecf0, 
args=0xcc6cd0, nargs=0, kwnames=0x0)
at /usr/src/debug/Python-3.7.3/Objects/call.c:730
#11 0x779cbf2c in call_function (pp_stack=0x7fff78b8, oparg=0, 
kwnames=0x0) at /usr/src/debug/Python-3.7.3/Python/ceval.c:4568
#12 0x779c6344 in _PyEval_EvalFrameDefault (f=0xcc6b08, throwflag=0) at 
/usr/src/debug/Python-3.7.3/Python/ceval.c:3093
#13 0x779b8067 in PyEval_EvalFrameEx (f=0xcc6b08, throwflag=0) at 
/usr/src/debug/Python-3.7.3/Python/ceval.c:547
#14 0x778b3b1a in function_code_fastcall (co=0x7620bc00, 
args=0x764fef90, nargs=1, globals=0x764e6120)
at /usr/src/debug/Python-3.7.3/Objects/call.c:283
#15 0x778b4096 in _PyFunction_FastCallKeywords (func=0x7fffedb63488, 
stack=0x764fef88, nargs=1, kwnames=0x0)
at /usr/src/debug/Python-3.7.3/Objects/call.c:408
#16 0x779cc357 in call_function (pp_stack=0x7fff9140, oparg=1, 
kwnames=0x0) at /usr/src/debug/Python-3.7.3/Python/ceval.c:4616
#17 0x779c64c9 in _PyEval_EvalFrameDefault (f=0x764fee08, 
throwflag=0) at /usr/src/debug/Python-3.7.3/Python/ceval.c:3124
#18 0x779b8067 in PyEval_EvalFrameEx (f=0x764fee08, throwflag=0) at 
/usr/src/debug/Python-3.7.3/Python/ceval.c:547
#19 0x779ca14f in _PyEval_EvalCodeWithName (_co=0x7620bc90, 
globals=0x764e6120, locals=0x764e6120, args=0x0, argcount=0, 
kwnames=0x0, 
kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0, kwdefs=0x0, 
closure=0x0, name=0x0, qualname=0x0) at 
/usr/src/debug/Python-3.7.3/Python/ceval.c:3930
#20 0x779ca347 in PyEval_EvalCodeEx (_co=0x7620bc90, 
globals=0x764e6120, locals=0x764e6120, args=0x0, argcount=0, kws=0x0, 
kwcount=0, 
defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at 
/usr/src/debug/Python-3.7.3/Python/ceval.c:3959
#21 0x779b7ff1 in PyEval_EvalCode (co=0x7620bc90, 
globals=0x764e6120, locals=0x764e6120) at 
/usr/src/debug/Python-3.7.3/Python/ceval.c:524
#22 0x779b2963 in builtin_exec_impl (module=0x77e61c28, 
source=0x7620bc90, globals=0x764e6120, locals=0x764e6120)
at /usr/src/debug/Python-3.7.3/Python/bltinmodule.c:1079
#23 0x779aff3b in builtin_exec (module=0x77e61c28, args=0x6c5b58, 
nargs=2) at /usr/src/debug/Python-3.7.3/Python/clinic/bltinmodule.c.h:283
#24 0x778b48bd in _PyMethodDef_RawFastCallKeywords 
(method=0x77d7bae0 , self=0x77e61c28, 
args=0x6c5b58, nargs=2, 
kwnames=0x0) at /usr/src/debug/Python-3.7.3/Objects/call.c:651
#25 0x778b4bab in _PyCFunction_FastCallKeywords (func=0x76566d80, 
args=0x6c5b58, nargs=2, kwnames=0x0)
at /usr/src/debug/Python-3.7.3/Objects/call.c:730
#26 0x779cbf2c in call_function (pp_stack=0x7fffada0, oparg=2, 
kwnames=0x0) at /usr/src/debug/Python-3.7.3/Python/ceval.c:4568
#27 0x779c64c9 in _PyEval_Eval

[issue37433] syntax error in f-string in multiline string produces ~40k spaces of output

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37434] Segfault in typeobject.c at _PyObject_GC_UNTRACK(type)

[Tim Lusk]

Tim Lusk  added the comment:

Alright I was finally able to reproduce the issue with a small code snip and 
capture the stack in gdb.  It looks like the issue has something to do with an 
exception thrown while importing a class (and possibly only when the class is a 
c_ext, but I haven't confirmed that)

[root@933fdf38d838 ~]# python3.7 -m venv crash-test
[root@933fdf38d838 ~]# crash-test/bin/pip install protobuf
Looking in indexes: https://files.pythonhosted.org/simple
Collecting protobuf
  Downloading 
https://files.pythonhosted.org/packages/ce/86/9f6123c4c6f481862f286dbe13aa2e97bdedd7662f5fc3033c1a41f32f88/protobuf-3.8.0-cp37-cp37m-manylinux1_x86_64.whl
 (1.2MB)
100% || 1.2MB 2.3MB/s 
Collecting six>=1.9 (from protobuf)
  Downloading 
https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools in 
./crash-test/lib/python3.7/site-packages (from protobuf) (40.8.0)
Installing collected packages: six, protobuf
Successfully installed protobuf-3.8.0 six-1.12.0
[root@933fdf38d838 ~]# scl enable devtoolset-8 "gdb ./crash-test/bin/python"
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-2.el6
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./crash-test/bin/python...Reading symbols from 
/usr/lib/debug/usr/bin/python3.7m.debug...
warning: Skipping deprecated .gdb_index section in 
/usr/lib/debug/usr/bin/python3.7m.debug.
Do "set use-deprecated-index-sections on" before the file is read
to use the section anyway.
done.
done.
(gdb) run
Starting program: /root/crash-test/bin/python 
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.212.el6.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Python 3.7.3 (default, Jun 27 2019, 15:27:42) 
[GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import warnings
>>> import gc
>>> 
>>> gc.disable()
>>> 
>>> for i in range(100):
... warnings.filterwarnings('error')
... try:
... from google.protobuf.pyext import _message
... except Exception as ex:
... print(i)
... 
... gc.collect()
... 
0
1
2
3
4
5

Program received signal SIGSEGV, Segmentation fault.
0x77922ae5 in type_dealloc (type=0x72277480) at 
/usr/src/debug/Python-3.7.3/Objects/typeobject.c:3283
3283_PyObject_GC_UNTRACK(type);
Missing separate debuginfos, use: debuginfo-install 
bzip2-libs-1.0.5-7.el6_0.x86_64 expat-2.0.1-13.el6_8.x86_64 
libgcc-4.4.7-23.el6.x86_64 libstdc++-4.4.7-23.el6.x86_64 
ncurses-libs-5.7-4.20090207.el6.x86_64 
nss-softokn-freebl-3.14.3-23.3.el6_8.x86_64 
openresty-openssl-1.0.2s-1.el6.x86_64 readline-6.0-4.el6.x86_64 
xz-libs-4.999.9-0.5.beta.20091007git.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  0x77922ae5 in type_dealloc (type=0x72277480) at 
/usr/src/debug/Python-3.7.3/Objects/typeobject.c:3283
#1  0x778ee721 in free_keys_object (keys=0x764e3bc0) at 
/usr/src/debug/Python-3.7.3/Objects/dictobject.c:559
#2  0x778f20af in dict_dealloc (mp=0x747a1708) at 
/usr/src/debug/Python-3.7.3/Objects/dictobject.c:1913
#3  0x77904d15 in module_clear (m=0x722f3ea8) at 
/usr/src/debug/Python-3.7.3/Objects/moduleobject.c:749
#4  0x77a3aeda in delete_garbage (collectable=0x7fffc250, 
old=0x77dd75f0 <_PyRuntime+400>)
at /usr/src/debug/Python-3.7.3/Modules/gcmodule.c:769
#5  0x77a3b41d in collect (generation=2, n_collected=0x7fffc2b8, 
n_uncollectable=0x7fffc2c0, nofail=0)
at /usr/src/debug/Python-3.7.3/Modules/gcmodule.c:924
#6  0x77a3b845 in collect_with_callback (generation=2) at 
/usr/src/debug/Python-3.7.3/Modules/gcmodule.c:1036
#7  0x77a3bcef in gc_collect_impl (module=0x761f6138, generation=2)
at /usr/src/debug/Python-3.7.3/Modules/gcmodule.c:1138
#8  0x77a3ba13 in gc_collect (module=0x761f6138, 
args=0x764e7768, nargs=0, kwnames=0x0)
at /usr/src/debug/Python-3.7.3/Modules/clinic/gcmodule.c.h:100
#9  0x778b48e5 in _PyMethodDef_RawFastCallKeywords 
(method=0x77d8b800 , self=0x761f6138, 
args=0x764e7768, nargs=0, kwnames=0x0) at 
/usr/sr

[issue37434] Segfault in typeobject.c at _PyObject_GC_UNTRACK(type)

[Timothy Lusk]

Timothy Lusk  added the comment:

I've realized this might not be Python 3.7 specific, the reason protobuf is 
throwing an exception during import in my above example (as well as in our 
application) is because it's throwing a deprecation warning that was only 
introduced in 3.7.  

Python 3.6 doesn't cause any warnings importing protobuf so it doesn't abort 
the import.  Might need a different test case to confirm what versions this 
does apply to.

I've made a change to our application's error handling in this area so we 
aren't having a segfault anymore, but this is still a bug in Python itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37433] syntax error in f-string in multiline string produces ~40k spaces of output

[Anthony Sottile]

Anthony Sottile  added the comment:

I can also trigger this with:

```
1+ """\q
"""
```

And `python -Werror`

so it seems not necessarily related to fstrings

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37435] builtin map iterator should provide __length_hint__

[Nicholas Musolino]

New submission from Nicholas Musolino :

The builtin `map` function takes one or more user-supplied iterators, and 
returns an iterator.

When all the user-supplied iterator arguments to `map` provide a valid length 
hint (according to the PEP 424 length hint protocol), the iterator returned by 
`map` should provide a length hint.  

I suggest the following behavior:  when all the iterator arguments provide a 
valid length hint, the map iterator should return the minimum value among all 
length hint values, because it stops upon exhaustion of the shortest input 
iterator.

If any user-supplied iterator does *not* provide a length hint according to the 
PEP 424 protocol, the map iterator should return the `NotImplemented` 
singleton, in accordance with the protocol.

When the evaluation of `__length_hint__()` for a user-supplied iterator raises 
an exception, the exception should be propagated by the map iterator's 
`__length_hint__()` method.

--
components: Interpreter Core
messages: 346772
nosy: nmusolino
priority: normal
severity: normal
status: open
title: builtin map iterator should provide __length_hint__
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37433] syntax error in f-string in multiline string produces ~40k spaces of output

[Anthony Sottile]

Anthony Sottile  added the comment:

Debugging further, there's a token with a `multi_line_start` pointing at 
uninitialized memory -- let's see if I can't track down where that gets set and 
fix it :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36511] Add Windows ARM32 buildbot

[Paul Monson]

Change by Paul Monson :


--
pull_requests: +14247
pull_request: https://github.com/python/cpython/pull/14431

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37435] builtin map iterator should provide __length_hint__

[Nicholas Musolino]

Change by Nicholas Musolino :


--
keywords: +patch
pull_requests: +14248
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14432

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37435] builtin map iterator should provide __length_hint__

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This is a duplicate of issue26828.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37201] fix test_distutils failures for Windows ARM64

[Paul Monson]

Change by Paul Monson :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37236] fix test_complex for Windows arm64

[Paul Monson]

Change by Paul Monson :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36941] Windows build changes for Windows ARM64

[Paul Monson]

Change by Paul Monson :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36942] Windows code changes for Windows ARM64

[Paul Monson]

Change by Paul Monson :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com