date:20190412

[issue36593] isinstance check fails for Mock objects with spec executed under tracing function

2019-04-12 Thread Karthikeyan Singaravelan



Change by Karthikeyan Singaravelan :


--
title: Trace function interferes with MagicMock isinstance? -> isinstance check 
fails for Mock objects with spec executed under tracing function

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

2019-04-12 Thread David Bolen



David Bolen  added the comment:

I just noticed that my last message referenced the wrong commit.  My test 
failures were against commit f13c5c8b9401a9dc19e95d8b420ee100ac022208 (the same 
as Victor).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36602] Recursive directory list with pathlib.Path.iterdir

2019-04-12 Thread Laurie Opperman



Laurie Opperman  added the comment:

Having `iterdir(recursive=True)` recurse into symlinks to directories would 
mean we would either not yield those symlinks, or we yield those symlinks and 
all other directories.

I feel like not yielding directories is the way to go, but it's easy enough to 
check if a yielded path is a directory in application code.

The current implementation of using recursion to list subdirectory contents 
doesn't seem to allow for the obvious implementation of symlink 
cycle-detection: keeping track of which (real) directories have been listed.

PS: I've updated the pull-request to not follow symlinks to directories. This 
is not a final decision, but just updating to be in line with what I've implied 
up to this point

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33608] Add a cross-interpreter-safe mechanism to indicate that an object may be destroyed.

2019-04-12 Thread David Bolen



David Bolen  added the comment:

Eric, I'm also seeing the same Win7 and Win10 worker failures with commit 
b75b1a350 as last time (test_multiprocessing_spawn on both, and test_io on 
Win7).

For test_multiprocessing_spawn, it fails differently than Victor since no core 
file is generated, but I assume it's related in terms of child process 
termination.  See for example 
https://buildbot.python.org/all/#/builders/3/builds/2390 for Win10, where 
test_mymanager_context fails with:

==
FAIL: test_mymanager_context 
(test.test_multiprocessing_spawn.WithManagerTestMyManager)
--
Traceback (most recent call last):
  File 
"D:\buildarea\3.x.bolen-windows10\build\lib\test\_test_multiprocessing.py", 
line 2747, in test_mymanager_context
self.assertIn(manager._process.exitcode, (0, -signal.SIGTERM))
AssertionError: 3221225477 not found in (0, -15)
--

(3221225477 is C005 which I believe is an access violation)

For some reason, the Windows 7 worker didn't get a test run while your commit 
was live, but I can reproduce the same error manually.

For test_io, as before, its a shutdown lock failure:

==
FAIL: test_daemon_threads_shutdown_stdout_deadlock (test.test_io.CMiscIOTest)
--
Traceback (most recent call last):
  File "D:\cygwin\home\db3l\python.test\lib\test\test_io.py", line 4157, in 
test_daemon_threads_shutdown_stdout_deadlock
self.check_daemon_threads_shutdown_deadlock('stdout')
  File "D:\cygwin\home\db3l\python.test\lib\test\test_io.py", line 4148, in 
check_daemon_threads_shutdown_deadlock
self.assertIn("Fatal Python error: could not acquire lock "
AssertionError: "Fatal Python error: could not acquire lock for 
<_io.BufferedWriter name=''> at interpreter shutdown, possibly due to 
daemon threads" not found in ''

--

In manual attempts I have yet to be able to recreate the 
test_multiprocessing_spawn failure under Win10 but can usually manage a 25-50% 
failure rate under Win7 (which is much slower).  The test_io failure on Win7 
however, appears to be more easily reproducible.

It's possible I/O is more critical than CPU, or perhaps its impact on latency; 
I seem to more easily exacerbate the test_multiprocessing_spawn failure rate by 
loading down the host disk than its CPU.  I also noticed that the Win10 failure 
was when test_io and test_multiprocessing_spawn overlapped.

While I'm guessing this should happen on any low powered Windows VM, if it 
would help, I could arrange remote access to the Win7 worker for you.  Or just 
test a change on your behalf.  In fairness, it's unlikely to be useful for any 
significant remote debugging but perhaps at least having somewhere you could 
test a change, even if just with print-based debugging, might help.  And while 
it might be an independent issue, the test_io failure rate appears to occur 
more reliably than test_multiprocessing_spawn.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36601] signals can be caught by any thread

2019-04-12 Thread Nathaniel Smith

Nathaniel Smith added the comment:

Yeah, the check makes sense on a system like the comment says IRIX used to be,
where getpid() actually returns a thread id *and* signals are delivered to all
threads (wtf). But any modern system should do the POSIX thing, where getpid()
returns the same value in all threads, and signals are only delivered to one
thread anyway. So I agree with everyone else that the original motivation
doesn't make sense any more.

The place where this would change things is in fork/vfork context. For fork it
seems unhelpful... like Greg notes, we already reset main_pid in the AfterFork
handler, so the only effect is that there's a brief moment where signals can be
lost. If we removed the (getpid() == main_thread) check, then fork() would work
slightly better.

For vfork, man, I don't even know. Here I do disagree with Greg a little –
according to POSIX, trip_signal is *not* safe to call in a vfork'ed process.
POSIX says: "behavior is undefined if the process created by vfork() either
modifies any data [...] or calls any other function". Yes this is really as
extreme as it sounds – you're not allowed to mutate data or use any syscalls.
And they explicitly note that this applies to signal handlers too: "If signal
handlers are invoked in the child process after vfork(), they must follow the
same rules as other code in the child process."

trip_signals sets a flag -> it mutates data -> it technically can invoke
undefined behavior if it's called in a child process after a vfork. And it can
call write(), which again, undefined behavior.

Of course this is so restrictive that vfork() is almost unusable in Python
anyway, because you can't do anything in Python without modifying memory.

And worse: according to a strict reading of POSIX, vfork() should call
pthread_atfork() handlers, and our pthread_atfork() handlers mutate memory.

So from the POSIX-rules-lawyer perspective, there's absolutely no way any
Python process can ever call vfork() without invoking undefined behavior, no
matter what we do here.

Do we care?

It looks like subprocess.py was recently modified to call posix_spawn in some
cases:
https://github.com/python/cpython/blob/a304b136adda3575898d8b5debedcd48d5072272/Lib/subprocess.py#L610-L654

If we believe the comments in that function, it only does this on macOS – where
posix_spawn is a native syscall, so no vfork is involved – or on systems using
glibc (i.e., Linux), where posix_spawn *does* invoke vfork(). So in this one
case, currently, CPython does use vfork(). Also, users might call
os.posix_spawn directly on any system.

However, checking the glibc source, their implementation of posix_spawn
actually takes care of this – it doesn't use vfork() directly, but rather
clone(), and it takes care to make sure that no signal handlers run in the
child (see sysdeps/unix/sysv/linux/spawni.c for details).

AFAICT, the only ways that someone could potentially get themselves into
trouble with vfork() on CPython are:

- by explicitly wrapping it themselves, in which case, good luck to them I
guess. On Linux they aren't instantly doomed, because Linux intentionally
deviates from POSIX and *doesn't* invoke pthread_atfork handlers after vfork(),
but they still have their work cut out for them. Not really our problem.

- by explicitly calling os.posix_spawn on some system where this is implemented
using vfork internally, but with a broken libc that doesn't handle signals or
pthread_atfork correctly. Currently we don't know of any such systems, and if
they do exist they have to be pretty rare.

So: I think we shouldn't worry about vfork(), and it's fine to remove the
(getpid() == main_pid) check.

--
nosy: +njs

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36540] PEP 570: Python Positional-Only Parameters

2019-04-12 Thread Pablo Galindo Salgado



Pablo Galindo Salgado  added the comment:

I will separate the work in two Pull Request (maybe more). The implementation 
of the PEP will be done in PR12701 and the documentation and other additions 
will be done in future Pull Requests to keep things more manageable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27987] obmalloc's 8-byte alignment causes undefined behavior

2019-04-12 Thread Inada Naoki



Change by Inada Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36623] Clean unused parser headers

2019-04-12 Thread Pablo Galindo Salgado



New submission from Pablo Galindo Salgado :

After the removal of pgen, there are multiple parser headers that are not used 
anymore or ar lacking implementations.

--
components: Interpreter Core
messages: 340140
nosy: pablogsal
priority: normal
severity: normal
status: open
title: Clean unused parser headers
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36623] Clean unused parser headers

2019-04-12 Thread Pablo Galindo Salgado



Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +12742
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36622] Inconsistent exponent notation formatting

2019-04-12 Thread Eric V. Smith



Change by Eric V. Smith :


--
nosy: +eric.smith, mark.dickinson, skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

1 2 >

1 - 100 of 184 matches

Mail list logo