[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Leonardo Rick]

Leonardo Rick  added the comment:

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Program Files\Python39\lib\multiprocessing\spawn.py", line 116, in 
spawn_main
exitcode = _main(fd, parent_sentinel)
  File "C:\Program Files\Python39\lib\multiprocessing\spawn.py", line 126, in 
_main
self = reduction.pickle.load(from_parent)
  File "C:\Program Files\Python39\lib\multiprocessing\connection.py", line 967, 
in rebuild_pipe_connection
handle = dh.detach()
  File "C:\Program Files\Python39\lib\multiprocessing\reduction.py", line 131, 
in detach
return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Acesso negado

--
nosy: +leonardo2
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Jesvi Jonathan]

Jesvi Jonathan  added the comment:

File "c:/Users/jesvi/Documents/GitHub/Jesvi-Bot-Telegram/scripts/main.py", line 
144, in thread_test
p.start()
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py",
 line 121, in start
self._popen = self._Popen(self)
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py",
 line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py",
 line 327, in _Popen
return Popen(process_obj)
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py",
 line 93, in __init__
reduction.dump(process_obj, to_child)
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py",
 line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Traceback (most recent call last):
  File "", line 1, in 
Traceback (most recent call last):
  File "", line 1, in 
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py",
 line 107, in spawn_main
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py",
 line 107, in spawn_main
new_handle = reduction.duplicate(pipe_handle,
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py",
 line 79, in duplicate
new_handle = reduction.duplicate(pipe_handle,
  File 
"C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py",
 line 79, in duplicate
return _winapi.DuplicateHandle(
OSError: [WinError 6] The handle is invalid
return _winapi.DuplicateHandle(
OSError: [WinError 6] The handle is invalid

--
nosy: +jesvi22j
type: behavior -> compile error
versions:  -Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Eryk Sun]

Eryk Sun  added the comment:

spawn_main() could handle a PermissionError by checking the exit code. If the 
parent has terminated already, then simply exit quietly. 
_winapi.PROCESS_QUERY_LIMITED_INFORMATION would need to be defined. For example:

def spawn_main(pipe_handle, parent_pid=None, tracker_fd=None):
"""Run code specified by data received over a pipe."""
assert is_forking(sys.argv), "Not forking"

if sys.platform == 'win32':
import msvcrt
import _winapi

if parent_pid is not None:
source_process = _winapi.OpenProcess(
_winapi.SYNCHRONIZE |
_winapi.PROCESS_DUP_HANDLE |
_winapi.PROCESS_QUERY_LIMITED_INFORMATION,
False, parent_pid)
else:
source_process = None
try:
new_handle = reduction.duplicate(
pipe_handle, source_process=source_process)
fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY)
exitcode = _main(fd, source_process)
except PermissionError:
if (source_process is None or
  _winapi.GetExitCodeProcess(source_process) ==
  _winapi.STILL_ACTIVE):
raise
exitcode = 1
else:
from . import resource_tracker
resource_tracker._resource_tracker._fd = tracker_fd
exitcode = _main(pipe_handle, os.dup(pipe_handle))

sys.exit(exitcode)

--
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.10 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Yan Ren]

Yan Ren  added the comment:

append Nosy List.

--
nosy: +pemryan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Pedro Algarvio]

Pedro Algarvio  added the comment:

What I'm able to copy from the console(though I doubt I'm getting all of
the traceback):

Traceback (most recent call last):

  File "", line 1, in 

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py",
line 106, in spawn_main

exitcode = _main(fd)

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py",
line 116, in _main

self = pickle.load(from_parent)

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\connection.py",
line 942, in rebuild_pipe_connection

handle = dh.detach()

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\reduction.py",
line 128, in detach

self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE)

PermissionError: [WinError 5] Access is denied

Pedro Algarvio @ Phone

A sábado, 22/02/2020, 13:44, Eryk Sun  escreveu:

>
> Eryk Sun  added the comment:
>
> > I seem to be consistingly hitting this issue.
>
> It will help with test development if you can provide a minimal example
> that reliably reproduces the problem.
>
> In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED
> (5). Assuming the process handles have the required PROCESS_DUP_HANDLE
> access, it's most likely the case that the underlying NtDuplicateObject
> system call is failing with STATUS_PROCESS_IS_TERMINATING (0xC10A). For
> example:
>
> import ctypes
> ntdll = ctypes.WinDLL('ntdll')
> from subprocess import Popen, PIPE
> from _winapi import GetCurrentProcess, TerminateProcess
> from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS
>
> p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE)
> TerminateProcess(p._handle, 0)
>
> Try to duplicate the process handle into the terminated process:
>
> >>> source = GetCurrentProcess()
> >>> target = handle = p._handle
> >>> try:
> ... DuplicateHandle(source, handle, target,
> ... 0, False, DUPLICATE_SAME_ACCESS)
> ... except:
> ... status = ntdll.RtlGetLastNtStatus()
> ... print(f'NTSTATUS: {status & (2**32-1):#010x}')
> ... raise
> ...
> NTSTATUS: 0xc10a
> Traceback (most recent call last):
>   File "", line 2, in 
> PermissionError: [WinError 5] Access is denied
>
> --
> nosy: +eryksun
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Eryk Sun]

Eryk Sun  added the comment:

> I seem to be consistingly hitting this issue.

It will help with test development if you can provide a minimal example that 
reliably reproduces the problem. 

In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED (5). 
Assuming the process handles have the required PROCESS_DUP_HANDLE access, it's 
most likely the case that the underlying NtDuplicateObject system call is 
failing with STATUS_PROCESS_IS_TERMINATING (0xC10A). For example:

import ctypes
ntdll = ctypes.WinDLL('ntdll')
from subprocess import Popen, PIPE
from _winapi import GetCurrentProcess, TerminateProcess
from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS

p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE)
TerminateProcess(p._handle, 0)

Try to duplicate the process handle into the terminated process:

>>> source = GetCurrentProcess()
>>> target = handle = p._handle
>>> try:
... DuplicateHandle(source, handle, target,
... 0, False, DUPLICATE_SAME_ACCESS)
... except:
... status = ntdll.RtlGetLastNtStatus() 
... print(f'NTSTATUS: {status & (2**32-1):#010x}')
... raise
...
NTSTATUS: 0xc10a
Traceback (most recent call last):
  File "", line 2, in 
PermissionError: [WinError 5] Access is denied

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[Pedro Algarvio]

Pedro Algarvio  added the comment:

Any possible workaround for this issue?
I seem to be consistingly hitting this issue.

--
nosy: +s0undt3ch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38263] [Windows] multiprocessing: DupHandle.detach() race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE)

[STINNER Victor]

New submission from STINNER Victor :

On Windows, the multiprocessing DupHandle.detach() method has race condition on 
DuplicateHandle(DUPLICATE_CLOSE_SOURCE).

Error on duplicate():

Traceback (most recent call last):
  File "", line 1, in 
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py",
 line 107, in spawn_main
new_handle = reduction.duplicate(pipe_handle,
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py",
 line 79, in duplicate
return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access is denied

Example: bpo-34714


Error on detach():

Traceback (most recent call last):
  File "", line 1, in 
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py",
 line 117, in spawn_main
exitcode = _main(fd)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py",
 line 127, in _main
self = reduction.pickle.load(from_parent)
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\connection.py",
 line 951, in rebuild_pipe_connection
handle = dh.detach()
  File 
"D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py",
 line 133, in detach
self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

Example: bpo-34513

--
components: Library (Lib)
messages: 353064
nosy: pablogsal, pitrou, vstinner
priority: normal
severity: normal
status: open
title: [Windows] multiprocessing: DupHandle.detach() race condition on 
DuplicateHandle(DUPLICATE_CLOSE_SOURCE)
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com