[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Sorry for the delay.  I've now merged the PR for 3.9.  Since this is a slight 
behaviour change, and the original issue was easy to workaround, I won't 
backport it.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset c2ac4cf040ea950bf552d1e77bea613a1a5474fe by Christopher Hunt in 
branch 'master':
bpo-35727: Use exit code 0 on sys.exit() in multiprocessing.Process. (GH-11538)
https://github.com/python/cpython/commit/c2ac4cf040ea950bf552d1e77bea613a1a5474fe


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

Christopher Hunt  added the comment:

Any other concerns here?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

Christopher Hunt  added the comment:

> I believe the mentality behind multiprocessing.Process triggering an exit 
> code of 1 when sys.exit() is invoked inside its process is to indicate a 
> non-standard exit out of its execution.

Can I ask what this is based on? I did a pretty good amount of digging but 
didn't find any justification for it. It just seems like a simple oversight to 
me.

> There may yet be other side effects that could be triggered by having a 
> sys.exit(0) translate into an exit code of 0 from the Process's process -- 
> and we might not notice them with the current tests.

This is definitely a behavior change and will break any code that currently 
relies on `sys.exit(None)` or `sys.exit()` exiting with a non-zero exit code 
from a multiprocessing.Process. The fact that all documentation indicates that 
`sys.exit(None)` or `sys.exit()` results in a 0 exit code in normal Python 
(with no documentation on it related to multiprocessing) makes me think that 
any code relying on this behavior is subtly broken, however. Any impacted user 
can update their code and explicitly pass 1 to `sys.exit`, which should be 
forward and backwards compatible.

> Was there a particular use case that motivates this suggested change?

I have a wrapper library that invokes arbitrary user code and attempts to 
behave as if that code was executed in a vanilla Python process, to include 
propagating the correct exit code.

Currently I have a workaround here: 
https://github.com/chrahunt/quicken/blob/2dd00a5f024d7b114b211aad8a2618ec8f101956/quicken/_internal/server.py#L344-L353,
 but it would be nice to get rid of it in 5-6 years if this fix gets in and the 
non-conformant Python versions fall out of support. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Davin Potts]

Davin Potts  added the comment:

I believe the mentality behind multiprocessing.Process triggering an exit code 
of 1 when sys.exit() is invoked inside its process is to indicate a 
non-standard exit out of its execution.  There may yet be other side effects 
that could be triggered by having a sys.exit(0) translate into an exit code of 
0 from the Process's process -- and we might not notice them with the current 
tests.

Was there a particular use case that motivates this suggested change?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Cheryl Sabella]

Change by Cheryl Sabella :


--
nosy: +davin
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Emmanuel Arias]

Emmanuel Arias  added the comment:

The same behavior on 3.8 and 3.5

--
nosy: +eamanu
versions: +Python 3.5, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

Change by Christopher Hunt :


--
pull_requests: +11143, 11144
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

Change by Christopher Hunt :


--
pull_requests: +11143, 11144, 11145
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

Change by Christopher Hunt :


--
pull_requests: +11143
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

Change by Christopher Hunt :


--
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

[Christopher Hunt]

New submission from Christopher Hunt :

When a function is executed by a multiprocessing.Process and uses sys.exit,
the actual exit code reported by multiprocessing is different than would be
expected given the Python interpreter behavior and documentation. For example,
given:

from functools import partial
from multiprocessing import get_context
import sys


def run(ctx, fn):
p = ctx.Process(target=fn)
p.start()
p.join()
return p.exitcode


if __name__ == '__main__':
ctx = get_context('fork')
print(run(ctx, partial(sys.exit, 2)))
print(run(ctx, partial(sys.exit, None)))
print(run(ctx, sys.exit))

ctx = get_context('spawn')
print(run(ctx, partial(sys.exit, 2)))
print(run(ctx, partial(sys.exit, None)))
print(run(ctx, sys.exit))

ctx = get_context('forkserver')
print(run(ctx, partial(sys.exit, 2)))
print(run(ctx, partial(sys.exit, None)))
print(run(ctx, sys.exit))

when executed results in

$ python exit.py
2
1
1
2
1
1
2
1
1


but when Python itself is executed we see different behavior

$ for arg in 2 None ''; do python -c "import sys; sys.exit($arg)"; echo $?; 
done
2
0
0

The documentation states

> sys.exit([arg])
> ...
> The optional argument arg can be an integer giving the exit status
> (defaulting to zero), or another type of object.

The relevant line in multiprocessing 
(https://github.com/python/cpython/blame/1cffd0eed313011c0c2bb071c8affeb4a7ed05c7/Lib/multiprocessing/process.py#L307)
seems to be from the original pyprocessing module itself, and I could
not locate an active site that maintains the repository to see if there
was any justification for the behavior.

--
components: Library (Lib)
files: multiprocessing-exitcode-3.7.1.patch
keywords: patch
messages: 333531
nosy: chrahunt
priority: normal
severity: normal
status: open
title: sys.exit() in a multiprocessing.Process does not align with Python 
behavior
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7
Added file: 
https://bugs.python.org/file48045/multiprocessing-exitcode-3.7.1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com