[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e6cfdefa0c2f5bda177e49b228c2c7528f7c239c by Victor Stinner in 
branch 'master':
bpo-31510: Fix multiprocessing test_many_processes() on macOS (#3857)
https://github.com/python/cpython/commit/e6cfdefa0c2f5bda177e49b228c2c7528f7c239c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

STINNER Victor  added the comment:

Antoine: "It seems other people have similar issues: 
https://github.com/libuv/libuv/issues/1226 Perhaps we need to relax the test on 
OSX :-/"

Oh thanks for the confirmation. I proposed a patch to accept -SIGKILL on macOS: 
PR 3857.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +3837
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[Antoine Pitrou]

Antoine Pitrou  added the comment:

It seems other people have similar issues:
https://github.com/libuv/libuv/issues/1226

Perhaps we need to relax the test on OSX :-/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> It looks like a weak synchronization.

It is.  I don't remember exactly why I had to add this, I can't reproduce any 
issue without it anymore...

> the signal is sent before Python registered signal handlers?

Python signal handlers are not relevant here, we're sending SIGTERM which kills 
the process.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

STINNER Victor added the comment:

test_many_processes() is made in two steps. The bug occurs at the second step 
which calls proc.terminate() on processes. Code of the test:

@classmethod
def _sleep_some(cls):
time.sleep(100)

@classmethod
def _test_sleep(cls, delay):
time.sleep(delay)

def test_many_processes(self):
if self.TYPE == 'threads':
self.skipTest('test not appropriate for {}'.format(self.TYPE))

sm = multiprocessing.get_start_method()
N = 5 if sm == 'spawn' else 100

# Try to overwhelm the forkserver loop with events
procs = [self.Process(target=self._test_sleep, args=(0.01,))
 for i in range(N)]
for p in procs:
p.start()
for p in procs:
join_process(p)
for p in procs:
self.assertEqual(p.exitcode, 0)

procs = [self.Process(target=self._sleep_some)
 for i in range(N)]
for p in procs:
p.start()
time.sleep(0.001)  # let the children start...
for p in procs:
p.terminate()
for p in procs:
join_process(p)
if os.name != 'nt':
for p in procs:
self.assertEqual(p.exitcode, -signal.SIGTERM) # <--- HERE

I'm not sure about the "time.sleep(0.001)  # let the children start...". It 
looks like a weak synchronization. Maybe if the system is heavily loaded, the 
signal is sent before Python registered signal handlers? 1 ms seems short to 
start Python. On my Linux laptop, it's closer to 15 ms.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

STINNER Victor added the comment:

test_many_processes() test was added by Antoine Pitrou in bpo-30589. This test 
was mentionned in bpo-30703.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

STINNER Victor added the comment:

I failed to reproduce the bug on Linux.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31510] test_many_processes() of test_multiprocessing_spawn failed on x86-64 Sierra 3.x

[STINNER Victor]

New submission from STINNER Victor:

The process was killed (SIGKILL).

See also bpo-30356.

http://buildbot.python.org/all/builders/x86-64%20Sierra%203.x/builds/765/steps/test/logs/stdio

==
FAIL: test_many_processes 
(test.test_multiprocessing_spawn.WithProcessesTestProcess)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/_test_multiprocessing.py",
 line 505, in test_many_processes
self.assertEqual(p.exitcode, -signal.SIGTERM)
AssertionError: -9 != -15

--
components: Tests
messages: 302469
nosy: haypo
priority: normal
severity: normal
status: open
title: test_many_processes() of test_multiprocessing_spawn failed on x86-64 
Sierra 3.x
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com