[issue38893] broken container/selinux integration

[Christian Heimes]

Christian Heimes  added the comment:

>From the Github bug:

copy2() fails while copying extended attributes.

# python3
Python 3.7.4 (default, Aug 12 2019, 14:45:07) 
[GCC 9.1.1 20190605 (Red Hat 9.1.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shutil
>>> shutil.copy2('/tmp/some_file', '/relabel_bug/failure')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib64/python3.7/shutil.py", line 267, in copy2
copystat(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib64/python3.7/shutil.py", line 209, in copystat
_copyxattr(src, dst, follow_symlinks=follow)
  File "/usr/lib64/python3.7/shutil.py", line 165, in _copyxattr
os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/relabel_bug/failure'

The setxattr() fail is blocked SELinux:
type=AVC msg=audit(1573815617.682:1332): avc:  denied  { relabelto } for  
pid=3157530 comm="python3" name="failure" dev="loop1" ino=12 
scontext=system_u:system_r:container_t:s0:c552,c859 
tcontext=system_u:object_r:fusefs_t:s0 tclass=file permissive=0


Could you please provide name and value of the setxattr() call? I bet it's 
trying to setxattr 'security.selinux' extended file attribute.

--
nosy: +christian.heimes
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38021] pep425 tag for AIX is inadequate

[Michael Felt]

Michael Felt  added the comment:

a) - thanks Ned, for the kind words.

b) - the proposed (change to the tag) is "AIX.VRTL.YYWW.SZ".
"AIX" - in caps, to distinguish from current tag starting as "aix"
VRTL - 4 digit number, one digit for Version, one digit as Revision, and two 
digits for Technology Level
YYWW - builddata with two digits for year (7 becomes 07) and two digits for 
week of year.
SZ - 32, or 64 bit size of executable (or pointer).

The builddate is crucial as system updates are possible, in oslevel -s terms, 
from 6100-09-11-1810 to 7100-05-02-1810 but not from 6100-09-12-1846 to 
7100-05-02-1810. Using the proposed PEP425 tag the VRTL-YYWW numbers here would 
be: 6109-1810, 6109-1846 and 7105-1810.

The build-date of the executable is essential, as it specifies the lowest date 
of any AIX release that it is supported on. This is something CPython must 
provide, as it does for other platforms, in the form of 
sysconfig.get_config_var("AIX_BUILDDATE"). The VRTL values of the executable 
are already available via sysconfig.get_config_var("BUILD_GNU_TYPE") while SZ 
is typically determined from sys.maxsize.

Happy to answer additional questions - and write additional .rst documentation 
as desired.

e.g., an

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38881] unexpected behaviour of random.choices with zero weights

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38893] broken container/selinux integration

[Leif Middelschulte]

New submission from Leif Middelschulte :

It seems Python does not necessarily determine that it is running inside a 
container correctly.

This leads to broken/unexpected behavior when trying to copy files across 
filesytems using `copy2`.
This directly affects Python3 inside the official `fedora:latest` image.

Steps to reproduce the issue can be found here:
https://github.com/containers/container-selinux/issues/81

https://bugs.python.org/issue26328 *might* be related too.

--
components: IO
messages: 357248
nosy: Leif Middelschulte
priority: normal
severity: normal
status: open
title: broken container/selinux integration
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38863] Improve is_cgi() in http.server

[Dong-hee Na]

Change by Dong-hee Na :


--
nosy: +Rhodri James, ethan.furman -corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38874] asyncio.Queue: putting items out of order when it is full

[Junyeong Jeong]

Junyeong Jeong  added the comment:

Thanks for having an interest in this issue.

I thought asyncio.Queue guarantees the order of items as .put called after I
read the code. But it does not.

I attach poc code here.
In my machine, this program prints this message and exits.
$ python poc-oooput.py
num=1, last_read_num=0
num=33, last_read_num=1
Traceback (most recent call last):
  File "poc-oooput.py", line 47, in 
asyncio.run(main())
  File "/home/esrse/.pyenv/versions/3.7.3/lib/python3.7/asyncio/runners.py", 
line 43, in run
return loop.run_until_complete(main)
  File 
"/home/esrse/.pyenv/versions/3.7.3/lib/python3.7/asyncio/base_events.py", line 
584, in run_until_complete
return future.result()
  File "poc-oooput.py", line 44, in main
await stream.read()
  File "poc-oooput.py", line 20, in read
assert num == self._last_read_num + 1
AssertionError

This shows you that the 33rd item is put before the 2nd item.

--
Added file: https://bugs.python.org/file48740/poc-oooput.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33387] Simplify bytecodes for try-finally, try-except and with blocks.

[Gregory P. Smith]

Gregory P. Smith  added the comment:

sweet! =)

--
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

[Gregory P. Smith]

Gregory P. Smith  added the comment:

+1 to re-exposing a way to do PEP-523.

PEP-523 added a public API, we unintentionally hid it behind the mask of 
Py_BUILD_CORE_MODULE in 3.8.  We shouldn't remove PEP-523's abilities without a 
deprecation cycle.  But given the use cases tend to be "deep" knowledge places 
that can adapt to API change and version checks, I think re-exposing it without 
the need for Py_BUILD_CORE_MODULE is reasonable.

Are you proposing this for 3.8.1?

Or is 3.8 burned for this 3.6 & 3.7 feature :sadface: (but we know how to work 
around it - so if our RM says it is, we'll deal) and re-exposing in some manner 
it only happens in 3.9?

--
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

[Gregory P. Smith]

Change by Gregory P. Smith :


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24995] better exception message when an unsupported object is passed to `async for` (pep 492)

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


Removed file: 
https://bugs.python.org/file48739/TransactionHistory-79b28f7d5748ad20cb8afb107f00330adbcd6d81.csv

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24995] better exception message when an unsupported object is passed to `async for` (pep 492)

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components:  -2to3 (2.x to 3.x conversion tool)
type: resource usage -> enhancement
versions:  -Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38526] zipfile.Path has the wrong method name

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Closing as fixed since PRs are merged. Thanks Alex for the report and 
@Claudiu.Popa for the patch.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38881] unexpected behaviour of random.choices with zero weights

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Alternatively, we could raise an exception if the weight total isn't positive.  
Am not sure that has any real worth, but it could be done with only a small 
additional cost to the critical path.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38881] unexpected behaviour of random.choices with zero weights

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> When zero weights are given, the last element of a sequence 
> is always chosen.

Given non-sensical input, that behavior is as reasonable as any other (fwiw, 
the same is also observed with all negative weights, even if the negative 
weights are unequal).

The documentation currently says, "weights are assumed to be non-negative."  
Perhaps it should say, "weights are assumed to be non-negative and have at 
least one positive weight."

--
assignee:  -> rhettinger
components: +Documentation -Library (Lib)
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38881] unexpected behaviour of random.choices with zero weights

[Raymond Hettinger]

Change by Raymond Hettinger :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38882] IDLE should not make the about dialog be a transient of the withdrawn root window

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The NMT doc adds "A transient window always appears in front of its parent." 
and I believe that this is why we use it and why I would not want to 
unconditionally delete the calls without thorough testing on both Linux and 
Windows.

IDLE calls this function for configdialog, its config-key subdialog, all query, 
search, and textview windows, help-about, and the IDLE-help doc window.  I 
believe in all cases the parent is normally an editor-base window.  There are 
also the tk/inter color and save dialogs.  The same may be true for them, but 
ultimately all should be tested both with menu clicks and hot keys.

Configdialog and about-IDLE are special in that their menu entries are 
transferred to the 'IDLE' submenu that macOS adds to the app menu.  So 
macosx.py has event handlers that make root the parent.  Both of these popup, 
on my Macbook, to the right of the centerline.  From what you say, at least 
these two should have conditional transient calls.

For reasons I don't understand (obsolete history?), there is also a 
mac-specific root-parent event handler for IDLE-help, even though that is left 
on the Help submenu, and popup up on top the the current window.

I can only test on my Macbook with installed Python (3.7.5, 3.8.0) and tk 
8.6.9.  Are all tk versions earlier than 8.6.10 the same?

--
components: +macOS
nosy: +ned.deily, ronaldoussoren, taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Change by Valentyn Tymofieiev :


--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38890] A subprocess.Popen created with creationFlags=DETACHED_PROCESS on Windows should not emit a ResourceWarning

[Bernie Hackett]

Change by Bernie Hackett :


--
nosy: +behackett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38892] Audit Hook doc typos and confusion

[Terry J. Reedy]

New submission from Terry J. Reedy :

Some suggestions for
https://docs.python.org/3.9/c-api/sys.html#c.PySys_AddAuditHook
https://docs.python.org/3.9/library/sys.html#sys.addaudithook

"Adds to the collection of active auditing hooks"
"Adds the callable hook to the collection of active auditing hooks"

Change 'Adds' to 'Add' or maybe 'Append' (see near below).
Insert 'the callable hook' in PySys version.
Change 'collection' to 'sequence' or 'list' (and change verb to 'append') since 
order is important.

PySys version:

I think the 'userData' explanation should be closer to the top.  

Grammar: "Functions in the runtime and standard library that raise events 
include the details in each function’s documentation and listed in the audit 
events table." is not a proper sentence.  Its 'skeleton' is "Functions ... 
include the details ... and listed ... .  

Either change 'include the details' to 'are detailed' and 'listed' to 'are 
listed' or split into two sentences:

"Functions in the runtime and standard library that raise events are  listed in 
the audit events table.  Details are in each function’s documentation.

Both, again:

"raises a auditing event".  'a' should be  'an'.  To me, this is slightly 
confusing because Python raises exceptions, but auditing events are not 
exceptions and do not normally abort execution.  Perhaps "This call is a 
'sys.addaudithook' event with no arguments that triggers an audit call."


https://docs.python.org/3.9/c-api/sys.html#c.PySys_Audit
https://docs.python.org/3.9/library/sys.html#sys.audit

Change 'Raises' to 'Raise'.


https://docs.python.org/3.8/library/audit_events.html

On pydev, Steve said "(though some won't be raised until 3.8.1... we should 
probably mark those, or at least update that page to warn that events may have 
been added over time)."

--
assignee: docs@python
components: Documentation, Interpreter Core
messages: 357240
nosy: christian.heimes, docs@python, steve.dower, terry.reedy
priority: normal
severity: normal
stage: needs patch
status: open
title: Audit Hook doc typos and confusion
type: behavior
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38891] ShareableList read and write access is O(N), should be O(1)

[Jake Northey]

New submission from Jake Northey :

For an illustration of the performance implications of the __getitem__ and 
__setitem__ implementation, see the article below.
https://medium.com/@rvprasad/performance-of-system-v-style-shared-memory-support-in-python-3-8-d7a7d1b1fb96

The issue appears to be due to the summing of ShareableList item sizes to 
generate an offset for the requested item.  Perhaps an offset-based index could 
be created in which the allocation sizes could be constructed by comparing two 
offets.

--
components: Library (Lib)
messages: 357239
nosy: Jake Northey, davin
priority: normal
severity: normal
status: open
title: ShareableList read and write access is O(N), should be O(1)
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24995] better exception message when an unsupported object is passed to `async for` (pep 492)

[SAKCHAI SUKWISET]

Change by SAKCHAI SUKWISET :


--
components: +2to3 (2.x to 3.x conversion tool)
type: enhancement -> resource usage
versions: +Python 2.7, Python 3.7, Python 3.8, Python 3.9
Added file: 
https://bugs.python.org/file48739/TransactionHistory-79b28f7d5748ad20cb8afb107f00330adbcd6d81.csv

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38889] Segmentation fault when using EPF Importer

[Eric V. Smith]

Eric V. Smith  added the comment:

I agree this doesn't look like a python bug.

However, if the original poster can reproduce it with a short example with no 
third party code, we could take another look. If so, please re-open this issue.

And just because the code worked on a different version of python doesn't mean 
there's no bug in the code. I've written plenty of code where errors were 
exposed in my code when updating python.

--
nosy: +eric.smith
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38890] A subprocess.Popen created with creationFlags=DETACHED_PROCESS on Windows should not emit a ResourceWarning

[Shane Harvey]

New submission from Shane Harvey :

In https://bugs.python.org/issue26741 Popen was changed to emit a 
ResourceWarning in __del__ if the process is still running. However, when 
running a daemon/detached process it is completely valid to delete a Popen 
object before the process is complete.

On Windows, a daemon process can be created by using the DETACHED_PROCESS 
option, eg:
import subprocess

DETACHED_PROCESS = getattr(subprocess, 'DETACHED_PROCESS', 0x0008)
popen = subprocess.Popen(args, creationflags=DETACHED_PROCESS)
print('Running daemon process: ', popen.pid)
del popen

Unfortunately, the above will emit the warning:
C:\python\Python37\lib\subprocess.py:839: ResourceWarning: subprocess 840 is 
still running
  ResourceWarning, source=self)
Running daemon process:  3212

This makes it complicated to run a daemon process without emitting the resource 
warning. The best solution I've got is to manually set the Popen.returncode to 
circumvent the warning, ie:

popen = subprocess.Popen(args, creationflags=DETACHED_PROCESS)
print('Running daemon process: ', popen.pid)
# Set the returncode to avoid erroneous warning:
# "ResourceWarning: subprocess XXX is still running".
popen.returncode = 0
del popen


Running the attached script on Windows only one warning is emitted:
$ python.exe -Wall test_daemon_win.py
C:\python\Python37\lib\subprocess.py:839: ResourceWarning: subprocess 3584 is 
still running
  ResourceWarning, source=self)
Running daemon process:  3584
Running daemon process:  1012

I propose that Popen should not raise the resource warning when the 
creationFlags includes DETACHED_PROCESS.

--
components: Library (Lib)
files: test_daemon_win.py
messages: 357237
nosy: ShaneHarvey
priority: normal
severity: normal
status: open
title: A subprocess.Popen created with creationFlags=DETACHED_PROCESS on 
Windows should not emit a ResourceWarning
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file48738/test_daemon_win.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38021] pep425 tag for AIX is inadequate

[Ned Deily]

Ned Deily  added the comment:

Brett, sorry I wasn't more explicit about my concerns here. I added you and 
Nick to this issue and to its PR not primarily to do do code review but because 
I think it brings up issues that I believe haven't been dealt with before in 
our evolving environment and may need guidance from the Steering Council and 
the PyPA, both of which you two have been involved in. The PR is about defining 
new platform tags, something which has potential impact across both CPython 
development (it can affect how Python itself is built) and across the packaging 
realm (as any packaging tool like, presumably, pip and wheel and others are 
potentially affected).  This also is relevant to recent discussions about the 
roles of PyPA and CPython development.  Because it has the portential to impact 
and require changes across the board, we need to make sure that proposed 
changes like this get adequate review and consensus across the board.  I'm not 
exactly sure how that should work so I think it's something that i
 s worth clarifying.  AIX is already a supported platform, at least on best 
effort basis thanks in large part to the efforts of non-core developers like 
Michael here (and thank you, Michael). So, in this case, the impact is probably 
most on the PyPA side so I think there needs to be their explicit involvement.  
But, for example, there is a risk of a similar PR adding new platform support 
without adequate review which could have a major long-term impact on CPython 
maintenance and support (and such things have happened in the past).  Perhaps 
we need to formulate an explicit policy about approval of changes to cpython 
that affect platform support and thus packaging.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36225] Lingering subinterpreters should be implicitly cleared on shutdown

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

Am abit swamped and sick atm. You can go on and submit a fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36854] GC operates out of global runtime state.

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +16818
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/17331

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

The behavior changes between Python 3.2 (no deadlock) and Python 3.3 
(deadlock). Deadlock also reproducible on Python 3.8.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38889] Segmentation fault when using EPF Importer

[Sebastian Szwarc]

Sebastian Szwarc  added the comment:

Previous python version didnt do segmentation fault.
Code is the same as before and it worked fine previously ergo code is
correct and this is python interpreter issue

On Thu, Nov 21, 2019 at 11:45 PM STINNER Victor  wrote:
>
>
> STINNER Victor  added the comment:
>
> It looks like a bug in EPF Importer: in ingestIncremental(), line 206 of 
> EPFIngester.py.
>
> I don't think that it's a bug in Python. I suggest to clos ethe issue.
>
> --
> nosy: +vstinner
>
> ___
> Python tracker 
> 
> ___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38889] Segmentation fault when using EPF Importer

[STINNER Victor]

STINNER Victor  added the comment:

It looks like a bug in EPF Importer: in ingestIncremental(), line 206 of 
EPFIngester.py.

I don't think that it's a bug in Python. I suggest to clos ethe issue.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38858] new_interpreter() should reuse more Py_InitializeFromConfig() code

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +16817
pull_request: https://github.com/python/cpython/pull/17330

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38889] Segmentation fault when using EPF Importer

[Sebastian Szwarc]

Sebastian Szwarc  added the comment:

PYTHONFAULTHANDLER=1 python EPFImporter_debug.py incremental/*
2019-11-21 23:34:09,787 [INFO]: Beginning import for the following directories:
incremental/itunes20191120
incremental/pricing20191120
2019-11-21 23:34:09,787 [INFO]: Importing files in incremental/itunes20191120
2019-11-21 23:34:09,788 [INFO]: Starting import of 
/home/sebastian/appromocje/incremental/itunes20191120...
2019-11-21 23:34:09,813 [INFO]: Beginning incremental ingest of 
epf_artist_video (31664 records)
Fatal Python error: Segmentation fault

Current thread 0x7f51058df740  (most recent call first):
  File "/home/sebastian/appromocje/EPFIngester.py", line 206 in 
ingestIncremental
  File "/home/sebastian/appromocje/EPFIngester.py", line 111 in ingest
  File "EPFImporter_debug.py", line 222 in doImport
  File "EPFImporter_debug.py", line 437 in main
  File "EPFImporter_debug.py", line 454 in 
Segmentation fault

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

input_deadlock.py in issue38884.zip has a left-over comment:
# Requires pip install tensorflow==2.0.0 tensorflow-transform==0.15.0 
Please ignore that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38889] Segmentation fault when using EPF Importer

[Sebastian Szwarc]

New submission from Sebastian Szwarc :

Python 2.7 on Ubuntu 18.04 LTS is not in latest version.

Both version 2.7.14 and 15 RC1 bringing segmentation fault error.
Code is the same and previously there was no error.
COde is EPFImporter.py tool written by Apple to handle importing of their 
database dumps.

--
components: Interpreter Core
messages: 357229
nosy: Sebastian Szwarc
priority: normal
severity: normal
status: open
title: Segmentation fault when using EPF Importer
type: crash
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16576] ctypes: structure with bitfields as argument

[Vinay Sajip]

Change by Vinay Sajip :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

Repro: unpack issue38884.zip
python input_deadlock.py

(fails on Python 3.7, but not on 2.7).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

Attaching a minimal repro with simple project hierarchy.

--
nosy:  -brett.cannon, eric.snow, ncoghlan
Added file: https://bugs.python.org/file48737/issue38884.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38765] `ast.AST._attributes` is used by `ast.dump()` but not documented

[Pekka Klärck]

Pekka Klärck  added the comment:

I'd say `_attributes` is already exposed as defining it in your own node 
affects many of the functions in the ast module. For example, `ast.dump(node, 
include_attributes=True)` makes no sense otherwise.

Whatever was the reason for the leading underscore must be the same reason that 
added the underscore to `_fields`. That attribute is already documented.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38888] Popen should use pidfd_open to implement a non-busy wait

[Роман Донченко]

Роман Донченко  added the comment:

Right, of course. I keep confusing it with timerfd_create.

--
title: Popen should use pidfd_create to implement a non-busy wait -> Popen 
should use pidfd_open to implement a non-busy wait

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38888] Popen should use pidfd_create to implement a non-busy wait

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +benjamin.peterson, gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16576] ctypes: structure with bitfields as argument

[Vinay Sajip]

Vinay Sajip  added the comment:


New changeset 91c15a542cb780377dcde8fc17ba93111bc1d1cf by Vinay Sajip (Miss 
Islington (bot)) in branch '3.7':
[3.7] bpo-16576: Add checks for bitfields passed by value to functions. 
(GH-17097) (GH-17224)
https://github.com/python/cpython/commit/91c15a542cb780377dcde8fc17ba93111bc1d1cf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38888] Popen should use pidfd_create to implement a non-busy wait

[STINNER Victor]

STINNER Victor  added the comment:

os.pidfd_open() was added to Python 3.9. Do you mean pidfd_open()?

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38888] Popen should use pidfd_create to implement a non-busy wait

[Роман Донченко]

New submission from Роман Донченко :

Popen.wait(timeout) is currently implemented on Unix-like systems using a busy 
wait, since the waitpid system call doesn't have a timeout argument.

On Linux, it's now possible to do better than that. You can create a PID file 
descriptor using pidfd_create and poll that descriptor with the specified 
timeout. Popen.wait should make use of that.

--
components: Library (Lib)
messages: 357222
nosy: SpecLad
priority: normal
severity: normal
status: open
title: Popen should use pidfd_create to implement a non-busy wait
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38526] zipfile.Path has the wrong method name

[miss-islington]

miss-islington  added the comment:


New changeset 107ed88cde3ae6f1cb01ae75575ea0f92c138464 by Miss Islington (bot) 
in branch '3.8':
bpo-38526: Fix zipfile.Path method name to be the correct one (GH-17317)
https://github.com/python/cpython/commit/107ed88cde3ae6f1cb01ae75575ea0f92c138464


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38526] zipfile.Path has the wrong method name

[miss-islington]

Change by miss-islington :


--
pull_requests: +16816
pull_request: https://github.com/python/cpython/pull/17329

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38887] test_asyncio: test_pipe_handle() failed on AMD64 Windows7 SP1 3.x

[STINNER Victor]

New submission from STINNER Victor :

https://buildbot.python.org/all/#/builders/40/builds/3465

ERROR: test_pipe_handle (test.test_asyncio.test_windows_utils.PipeTests)
--
Traceback (most recent call last):
  File 
"C:\buildbot.python.org\3.x.kloth-win64\build\lib\test\test_asyncio\test_windows_utils.py",
 line 78, in test_pipe_handle
raise RuntimeError('expected ERROR_INVALID_HANDLE')
RuntimeError: expected ERROR_INVALID_HANDLE

--
components: Tests
messages: 357220
nosy: vstinner
priority: normal
severity: normal
status: open
title: test_asyncio: test_pipe_handle() failed on AMD64 Windows7 SP1 3.x
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38721] modulefinder should use import hooks properly

[Dominic Littlewood]

Dominic Littlewood <11dlittlew...@gmail.com> added the comment:

You are correct about modulefinder being static. I read "run_script" and 
thought it was running a script, but it turns out I was being silly.

I shall correct the sys.path issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38526] zipfile.Path has the wrong method name

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 65444cf7fe84d8ca1f9b51c7f5992210751e08bb by Jason R. Coombs 
(Claudiu Popa) in branch 'master':
bpo-38526: Fix zipfile.Path method name to be the correct one (#17317)
https://github.com/python/cpython/commit/65444cf7fe84d8ca1f9b51c7f5992210751e08bb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

[Kyle Stanley]

Kyle Stanley  added the comment:

> some platforms apparently do have SO_REUSEPORT defined but the option still 
> doesn't work, resulting in a ValueError exception from 
> create_datagram_endpoint().

Are you aware of what currently supported platforms have SO_REUSEPORT defined 
where the *reuse_port* parameter doesn't actually work? This would be quite 
helpful to know. 

AFAIK, it's available and supported on the majority of Unix platforms (such as 
Linux 3.9+, MacOS and BSD). On Windows, SO_REUSEPORT is undefined.

Also, at the very least, the ValueError exception would occur is highly 
informative and specific:

raise ValueError('reuse_port not supported by socket module, '
 'SO_REUSEPORT defined but not implemented.')

> I just have a #ifdef block for every OS and don't even try to abstract it.

The main issue with this approach for our purposes is that we don't directly 
support every single available platform, as making the code specific to an OS 
incurs a long-term maintenance cost. See 
https://www.python.org/dev/peps/pep-0011/ for more details. Indirect patches 
are fine, but we only add platform-specific code if it is officially supported: 

"Patches which add platform-specific code such as the name of a specific 
platform to the configure script will generally not be accepted without the 
platform having official support."

IIUC, this applies to any code that explicitly defines the name of a specific 
platform. Official support not only involves us having someone with commit 
privileges that will provide it, but also a buildbot in place (or provided) and 
having the OS *version* supported upstream. For example, I think Linux kernel 
versions prior to 3.9 would fall under the category of "unsupported", since 
they are not supported upstream.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38883] Path.home() should ignore HOME env var like os.path.expanduser()

[Brett Cannon]

Change by Brett Cannon :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38870] Expose ast.unparse in the ast module

[Brett Cannon]

Change by Brett Cannon :


--
nosy: +brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38721] modulefinder should use import hooks properly

[Brandt Bucher]

Brandt Bucher  added the comment:

See prior discussion on this here:

https://github.com/python/cpython/pull/11787#discussion_r256442282

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38721] modulefinder should use import hooks properly

[Brandt Bucher]

Brandt Bucher  added the comment:

I'm not sure what you mean when you say "modulefinder currently will detect 
modules imported by a script dynamically by running the script"... modulefinder 
is completely static, no?

I also think that changing sys.path like this is a bad idea (I tried this in an 
earlier PR and was told to remove it and rework my solution). We lose 
thread-safety, and it has effects far outside of the scope of modulefinder 
(i.e. the hooks we call could cache it or something, and break later imports). 
If changing sys.path is the only way to accomplish this (still not exactly sure 
what), I doubt it will be accepted. It seems like this PR does a lot of 
unnecessary refactoring too, on first glance.

With that said, adding support for hooks should be fairly straightforward by 
just manually walking down sys.path/meta_path/path_hooks in _find_module... 
maybe.

--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Brett Cannon]

Brett Cannon  added the comment:

Can you test this with a package that isn't tensorflow (e.g. something else in 
the stdlib or something created manually; basically a smaller reproducer)? It's 
hard to diagnose if this is really Python or not without a smaller reproducer 
as tensorflow might be doing some things behind the scenes that would break 
this due to it being an extension module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38878] os.PathLike subclasshook causes subclass checks true on abstract implementation

[Brett Cannon]

Brett Cannon  added the comment:

Ah, your `assert` call threw me since it does succeed so it isn't acting as a 
test case.

--
resolution: not a bug -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Brett Cannon]

Change by Brett Cannon :


--
nosy: +brett.cannon, eric.snow, ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

[Brett Cannon]

Brett Cannon  added the comment:

Posted 
https://mail.python.org/archives/list/python-...@python.org/thread/4UZJYAZL3NHRAGN5WAMJC4IHAHEXF3QF/
 to see if anyone else wants to weigh in.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38886] permissions too restrictive in zipfile.writestr

[Raphael Dussin]

Change by Raphael Dussin :


--
keywords: +patch
pull_requests: +16814
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17327

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37334] Add a cancel method to asyncio Queues

[Andrew Svetlov]

Andrew Svetlov  added the comment:

1. Suppose we have 2 concurrent producers, a single queue and a consumer.
2. The first producer puts several items into the queue and then calls q.close()
3. The second producer also puts several items. It doesn't matter the second 
producer closes the queue at the end or not.
4. The consumer gets items from the queue and prints them.

What items are printed and what are canceled/skipped? With the proposed PR it 
depends on timings of putting and getting data items and the queue size. 
The output can vary from zero to all pushed data.
That's why I think that the idea is not reliable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38886] permissions too restrictive in zipfile.writestr

[Raphael Dussin]

New submission from Raphael Dussin :

zipfile.writestr write with permissions 600 by default and there is no way to 
override permissions. This can be an issue when writing zip archive of data one 
want to share (e.g. zarr zipstore, see 
https://github.com/zarr-developers/zarr-python/pull/517)


Proposed fix in upcoming PR is to add the desired permission as an optional arg.

--
components: Library (Lib)
messages: 357210
nosy: rdussin
priority: normal
severity: normal
status: open
title: permissions too restrictive in zipfile.writestr
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38878] os.PathLike subclasshook causes subclass checks true on abstract implementation

[Bar Harel]

Bar Harel  added the comment:

Hey Brett, that's exactly the bug. It's supposed to be False ofc.

On Thu, Nov 21, 2019, 9:45 PM Brett Cannon  wrote:

>
> Brett Cannon  added the comment:
>
> I can't reproduce in Python 3.8.0:
>
> >>> import os
> >>> class A(os.PathLike): pass
> ...
> >>> class B:
> ... def __fspath__(self): pass
> ...
> >>> issubclass(B, A)
> True
>
> Did you check against an older version of Python?
>
> --
> resolution:  -> not a bug
> stage:  -> resolved
> status: open -> closed
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

[Brett Cannon]

Brett Cannon  added the comment:

@Mark

> PEP 523 makes no mention of adding a getter or setter.
> Adding them is a big change to Python semantics and shouldn't, IMO, be done 
> without a PEP that explicit states they are going to be added.

Adding getters or setters for something that was previously doable that we 
accidentally took away from users is not an expansion of semantics; it's fixing 
a backwards-compatibility break in a way that lets us keep the goal of making 
PyInterpreterState opaque.

I'm considering this issue at a stalemate and so I'm going to loop in 
python-dev to help settle this. If that still locks up then we can bring this 
to the steering council.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38721] modulefinder should use import hooks properly

[Dominic Littlewood]

Change by Dominic Littlewood <11dlittlew...@gmail.com>:


--
keywords: +patch
pull_requests: +16813
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17326

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38885] Have os.PathLike inherit from typing.Protocol

[Brett Cannon]

Brett Cannon  added the comment:

Closing as importing 'typing' in 'os' is probably too much and would impact 
start-up time.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38885] Have os.PathLike inherit from typing.Protocol

[Brett Cannon]

New submission from Brett Cannon :

Since os.PathLike explicitly defines a a protocol, it would make sense to have 
it inherit from typing.Protocol instead of abc.ABC.

--
components: Library (Lib)
messages: 357206
nosy: brett.cannon
priority: normal
severity: normal
status: open
title: Have os.PathLike inherit from typing.Protocol
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38878] os.PathLike subclasshook causes subclass checks true on abstract implementation

[Brett Cannon]

Brett Cannon  added the comment:

I can't reproduce in Python 3.8.0:

>>> import os
>>> class A(os.PathLike): pass
...
>>> class B:
... def __fspath__(self): pass
...
>>> issubclass(B, A)
True

Did you check against an older version of Python?

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37334] Add a cancel method to asyncio Queues

[Yury Selivanov]

Yury Selivanov  added the comment:

This seems like a useful idea. I recommend to write a test implementation and 
play with it.


Andrew:
> I think the proposal makes the queues API more error-prone: concurrent put() 
> and close() produces unpredictable result on get() side.

How? Can you elaborate?


Caleb:
> I'm interested in how this change would affect the pattern of shutting down a 
> queue-processing task.

Agree, this can be useful for that.


Martin:
> Given the reactions I gather "close" is a better name for the method, so I 
> changed it accordingly.

Not sure I like "close" since it will *cancel* all getters and putters & 
discard all items in the queue AND allow further operation on the queue.  The 
latter part is really questionable -- what's the point of losing the data in 
the queue and resuming it?  Seems like a mechanism for writing unreliable code, 
but perhaps you can give us an example where this is necessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38881] unexpected behaviour of random.choices with zero weights

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38021] pep425 tag for AIX is inadequate

[Brett Cannon]

Brett Cannon  added the comment:

I'm not in a good position to review distutils stuff.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

Possibly related: https://bugs.python.org/issue35943

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35943] PyImport_GetModule() can return partially-initialized module

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

Thanks. Is it possible that this issue and  https://bugs.python.org/issue38884 
are duplicates?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34572] C unpickling bypasses import thread safety

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

While investigating[1], I observe that certain unpickling operations, for 
example, Unpickler.find_class, remain not thread-safe in Python 3.7.5 and 
earlier versions that I tried. I have not tried 3.8, but cannot reproduce this 
error on Python 2. 

For example, attached find_class_deadlock.py which consistently fails with a 
deadlock on Python 3.7.5.

I opened https://bugs.python.org/issue38884, which may be causing these errors, 
since the failure mode is similar, and in at least some codepaths, find_class 
seems to be calling __import__ [2].

[1] https://issues.apache.org/jira/browse/BEAM-8651
[2] 
https://github.com/python/cpython/blob/4ffc569b47bef9f95e443f3c56f7e7e32cb440c0/Lib/pickle.py#L1426.

--
nosy: +Valentyn Tymofieiev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34572] C unpickling bypasses import thread safety

[Valentyn Tymofieiev]

Change by Valentyn Tymofieiev :


Added file: https://bugs.python.org/file48736/find_class_deadlock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

Valentyn Tymofieiev  added the comment:

Attached import_deadlock.py fails with   

...
File "", line 980, in _find_and_load
  File "", line 149, in __enter__
  File "", line 94, in acquire
_frozen_importlib._DeadlockError: deadlock detected by 
_ModuleLock('tensorflow_transform.tf_metadata.metadata_io')

under the same conditions.

--
Added file: https://bugs.python.org/file48735/import_deadlock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38884] __import__ is not thread-safe on Python 3

[Valentyn Tymofieiev]

New submission from Valentyn Tymofieiev :

Attached import_module_not_found.py consistently fails for me on Python 3.7.5 
and earlier Python 3 versions that I have tried with 

  File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
  File "/usr/lib/python3.7/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
  File "import_module_not_found.py", line 8, in t2
return __import__("tensorflow.estimator", level=0)
ModuleNotFoundError: No module named 'tensorflow.estimator

Threads in this example finish successfully if executed sequentially.

I have not tried higher versions of Python, but I cannot reproduce this on 
Python 2.7. 

Is this an expected behavior?

Thank you.

--
files: import_module_not_found.py
messages: 357198
nosy: Valentyn Tymofieiev
priority: normal
severity: normal
status: open
title: __import__ is not thread-safe on Python 3
versions: Python 3.7
Added file: https://bugs.python.org/file48734/import_module_not_found.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38883] Path.home() should ignore HOME env var like os.path.expanduser()

[Anthony Sottile]

Change by Anthony Sottile :


--
nosy: +Anthony Sottile

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38883] Path.home() should ignore HOME env var like os.path.expanduser()

[Batuhan]

Change by Batuhan :


--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38883] Path.home() should ignore HOME env var like os.path.expanduser()

[Batuhan]

Change by Batuhan :


--
nosy: +BTaskaya, steve.dower
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38883] Path.home() should ignore HOME env var like os.path.expanduser()

[Christoph Reiter]

New submission from Christoph Reiter :

In issue36264 os.path.expanduser() was changed to no longer use the HOME 
environment variable on Windows.

There are two more ways in the stdlib to get the user directory, 
pathlib.Path.home() and pathlib.Path.expanduser() which internally use 
gethomedir() which still uses the HOME environment variable:

https://github.com/python/cpython/blob/0aca3a3a1e68b4ca2d334ab5255dfc267719096e/Lib/pathlib.py#L255

Since they are documented to work the same as os.path.expanduser() they should 
be changed to no longer use HOME as well.

--
components: Library (Lib)
messages: 357196
nosy: lazka
priority: normal
severity: normal
status: open
title: Path.home() should ignore HOME env var like os.path.expanduser()
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36264] os.path.expanduser should not use HOME on windows

[Christoph Reiter]

Christoph Reiter  added the comment:

I've filed issue38883

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38882] IDLE should not make the about dialog be a transient of the withdrawn root window

[Marc Culler]

New submission from Marc Culler :

The soon-to-be-released Tcl/Tk 8.6.10 includes some changes to the macOS
port which cause the wm transient command to behave in the way that the
Tk manual says it should:

  "A transient window will mirror state changes in the master and
   inherit the state of the master when initially mapped."

This means that a transient of a withdrawn window will be withdrawn.  In
the macOS version of IDLE the about dialog is assigned as a transient
of the root window which is withdrawn, so the dialog does not appear on
the screen.  Consequently, activating the about menu will cause IDLE
to become unresponsive as it waits for an offscreen window to be closed.

Deleting line 28 in aboutDialog.py:

self.transient(parent)

makes the about dialog behave normally.

--
assignee: terry.reedy
components: IDLE
messages: 357195
nosy: culler, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE should not make the about dialog be a transient of the withdrawn 
root window
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36264] os.path.expanduser should not use HOME on windows

[Christoph Reiter]

Christoph Reiter  added the comment:

> Was pathlib forgotten here? Pathlib.home() is documented to return
> the same as expanduser("~") but it still prefers HOME instead of
> USERPROFILE.
> 
> Yes, it was forgotten (why doesn't it just use expanduser?). We
> should file a new bug for that.

I'll file one.

> > Note that this change has some effect on cygwin/mingw environments
> which all set HOME and now potentially lead to programs no longer
> being able to find their config files.
> 
> Firstly these are not supported environments, so it's not "rushed"
> for us to not preemptively consider them (though we'll happily merge
> most PRs that fix them without impacting supported environments).

Yeah, you're right, sorry, my comment wasn't appropriate.

> And I thought the idea was that they'd use posixpath as os.path
> rather than ntpath? Cygwin in particular, which provides the full
> environment. MinGW is a bit lighter to be closer to normal Windows
> behaviour, which would suggest that using the Windows variables is
> preferable.

I meant executing Python scripts with official Python in bash on Windows which 
sets HOME. But I just checked with "git for Windows" and it sets HOME to 
USERPROFILE anyway, so that might only affect cygwin/msys2 which have their own 
user dir.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36225] Lingering subinterpreters should be implicitly cleared on shutdown

[Lewis Gaul]

Lewis Gaul  added the comment:

I've put together a test along the lines of what Nick suggested, see the 
attached patch. Running this hits the Fatal 'remaining subinterpreters' error 
as expected:

```
   > ./Programs/_testembed test_bpo36225
--- Pass 0 ---
interp 0 <0x1A561A0>, thread state <0x1A56DD0>: id(modules) = 139707107673104
interp 1 <0x1A9D050>, thread state <0x1A87620>: id(modules) = 139707106987472
interp 2 <0x1B02210>, thread state <0x1AEC320>: id(modules) = 139706981531088
interp 0 <0x1A561A0>, thread state <0x1A56DD0>: id(modules) = 139707107673104
interp 3 <0x1B53740>, thread state <0x1AFD980>: id(modules) = 139706980408304
interp 4 <0x1BA3390>, thread state <0x1B3C7A0>: id(modules) = 139706979780944
interp 0 <0x1A561A0>, thread state <0x1A56DD0>: id(modules) = 139707107673104
Fatal Python error: PyInterpreterState_Delete: remaining subinterpreters
Python runtime state: finalizing (tstate=0x1a56dd0)

Aborted
```

I'm happy to look a bit further into the fix for this - Eric's pointers in this 
thread look useful to get started. @nanjekyejoannah did you get anywhere with 
this?

--
keywords: +patch
nosy: +LewisGaul
Added file: https://bugs.python.org/file48733/finalise-subinterps-test.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38857] AsyncMock issue with awaitable return_value/side_effect/wraps

[Lisa Roach]

Change by Lisa Roach :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36264] os.path.expanduser should not use HOME on windows

[Steve Dower]

Steve Dower  added the comment:

> Was pathlib forgotten here? Pathlib.home() is documented to return the same 
> as expanduser("~") but it still prefers HOME instead of USERPROFILE.

Yes, it was forgotten (why doesn't it just use expanduser?). We should file a 
new bug for that.

> Note that this change has some effect on cygwin/mingw environments which all 
> set HOME and now potentially lead to programs no longer being able to find 
> their config files.

Firstly these are not supported environments, so it's not "rushed" for us to 
not preemptively consider them (though we'll happily merge most PRs that fix 
them without impacting supported environments).

And I thought the idea was that they'd use posixpath as os.path rather than 
ntpath? Cygwin in particular, which provides the full environment. MinGW is a 
bit lighter to be closer to normal Windows behaviour, which would suggest that 
using the Windows variables is preferable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38859] AsyncMock says it raises StopIteration but that is Impossible

[Lisa Roach]

Lisa Roach  added the comment:

PR merged, thanks Jason!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38857] AsyncMock issue with awaitable return_value/side_effect/wraps

[Andrew Svetlov]

Andrew Svetlov  added the comment:


New changeset b2744c1be73f5af0d2dc4b952389efc90c8de94e by Andrew Svetlov (Lisa 
Roach) in branch '3.8':
[3.8] bpo-38857: AsyncMock fix for awaitable values and StopIteration fix [3.8] 
(GH-17269) (#17304)
https://github.com/python/cpython/commit/b2744c1be73f5af0d2dc4b952389efc90c8de94e


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38877] Python 3.9 build fails under Debian 9.11

[Ned Deily]

Ned Deily  added the comment:

What ./configure options did you use?  Did you do a make clean or equivalent 
between the two build attempts?  If you used --enable-shared, try without it.

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37838] typing.get_type_hints not working with forward-declaration and decorated functions

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37838] typing.get_type_hints not working with forward-declaration and decorated functions

[miss-islington]

miss-islington  added the comment:


New changeset 9458c5c42bbe5fb6ef2393c9ee66f012a2c13ca3 by Miss Islington (bot) 
in branch '3.8':
bpo-37838: get_type_hints for wrapped functions with forward reference 
(GH-17126)
https://github.com/python/cpython/commit/9458c5c42bbe5fb6ef2393c9ee66f012a2c13ca3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37838] typing.get_type_hints not working with forward-declaration and decorated functions

[miss-islington]

miss-islington  added the comment:


New changeset 30e5bd8471d7531d051796c01e8ede01ade883dc by Miss Islington (bot) 
in branch '3.7':
bpo-37838: get_type_hints for wrapped functions with forward reference 
(GH-17126)
https://github.com/python/cpython/commit/30e5bd8471d7531d051796c01e8ede01ade883dc


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38651] Add WolfSSL support

[Daniel Johnson]

Daniel Johnson  added the comment:

Thank you for the replied. I understand completely and I don't think it would 
be simple patches to try and use the compatibility layer. I have discovered 
that the WolfSSL compatibility layer doesn't support the full OpenSSL API.

However, I have found that I can build Python without OpenSSL and then use the 
Python API that WolfSSL provides and that gets us 90% of the way there. I have 
decided to pursue that instead of trying to add WolfSSL support directly into 
Python.

I'm going to close this issue for now.

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38881] unexpected behaviour of random.choices with zero weights

[Iza Romanowska]

New submission from Iza Romanowska :

Hi, 
When zero weights are given, the last element of a sequence is always chosen. 
Example: 

hits= []
for i in range(100):
hits.append(random.choices(["A","B","C","D"], [0, 0, 0, 0])[0])
print (set(hits))

>> {'D'}

I guess that most users would expect that in case of zero weights it will 
default into a random.choice behaviour and select one option at random since 
this is what happens in cases when all weights are equal. Alternatively, it 
should return an empty array if the assumption was that all choices have a zero 
probability of being selected. Either way, if it is consistently choosing one 
option, this may be potentially difficult to spot in situations when a sequence 
of weights all equal to zero only happen sporadically.

--
components: Library (Lib)
messages: 357185
nosy: IRomanowska
priority: normal
severity: normal
status: open
title: unexpected behaviour of random.choices with zero weights
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37838] typing.get_type_hints not working with forward-declaration and decorated functions

[miss-islington]

Change by miss-islington :


--
pull_requests: +16811
pull_request: https://github.com/python/cpython/pull/17324

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37838] typing.get_type_hints not working with forward-declaration and decorated functions

[miss-islington]

Change by miss-islington :


--
pull_requests: +16812
pull_request: https://github.com/python/cpython/pull/17325

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37838] typing.get_type_hints not working with forward-declaration and decorated functions

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:


New changeset 0aca3a3a1e68b4ca2d334ab5255dfc267719096e by Ivan Levkivskyi 
(benedwards14) in branch 'master':
bpo-37838: get_type_hints for wrapped functions with forward reference 
(GH-17126)
https://github.com/python/cpython/commit/0aca3a3a1e68b4ca2d334ab5255dfc267719096e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

[Fabio Zadrozny]

Fabio Zadrozny  added the comment:

@Mark

First you have to explain to me how you envision changing the method code 
reliably in the debugger... 

Import hooks don't work (they'd break with something as simple as the code 
below)

def method():
   a = 10

mod = reload(old_mod)
old_mod.method.__code__ = mod.method.__code__

using the tracing also doesn't work (it's too late to change the code)

Note: consider the reload just an example, not the only use case (say, the user 
could pickle code objects to remote execution and the debugger should still 
work).

Also, consider you have to change the bytecode of methods which are only 
internal to a function (and thus can't be accessed from the outside).

Then, if you have a reliable way to do it, how do you keep track of those code 
objects to reapply the patching when breakpoints change? What if it adds a 
breakpoint to a new method, how do you locate it? Creating strong references to 
methods isn't an option because it would prevent things from being garbage 
collected (and you'd have to track all objects containing code objects for it 
to be reliable).

As a note, pydevd does have some support for hot-reloading, but there are too 
many corner-cases and it doesn't work 100% for live coding (it's an unsolved 
problem is Python) -- and I can't really envision it working for regular 
breakpoints as there are too many corner cases to handle.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36264] os.path.expanduser should not use HOME on windows

[Christoph Reiter]

Christoph Reiter  added the comment:

Was pathlib forgotten here? Pathlib.home() is documented to return the same as 
expanduser("~") but it still prefers HOME instead of USERPROFILE.

Note that this change has some effect on cygwin/mingw environments which all 
set HOME and now potentially lead to programs no longer being able to find 
their config files. Not that I disagree with this change, just that it seems a 
bit rushed to me.

--
nosy: +lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38880] Subinterpreters: List interpreters associated with a channel end

[Lewis Gaul]

Change by Lewis Gaul :


--
keywords: +patch
pull_requests: +16810
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17323

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38880] Subinterpreters: List interpreters associated with a channel end

[Lewis Gaul]

New submission from Lewis Gaul :

The public interpreters API being implemented for PEP 554 requires the ability 
to list interpreters associated with channel ends. This functionality needs 
adding in the internal subinterpreters module.

See https://github.com/ericsnowcurrently/multi-core-python/issues/52 and 
https://www.python.org/dev/peps/pep-0554/#api-for-sharing-data.

--
messages: 357181
nosy: Lewis Gaul, eric.snow, nanjekyejoannah
priority: normal
severity: normal
status: open
title: Subinterpreters: List interpreters associated with a channel end
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38879] Reordered error checking in PyArena_New().

[Zoltán Szatmáry]

Change by Zoltán Szatmáry :


--
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com